2024-08-15T17:46:12.474 INFO:root:teuthology version: 1.2.0 2024-08-15T17:46:12.494 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-15T17:46:12.577 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038 branch: wip-mchangir-mgr-volumes-print-thread-name-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} email: mchangir@redhat.com first_in_suite: false job_id: '7857038' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - filesystem is online with fewer MDS than max_mds - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5995 sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 sleep_before_teardown: 0 suite: fs suite_branch: wip-mchangir-mgr-volumes-print-thread-name-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 targets: smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-15_16:39:12 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-15T17:46:12.578 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa; will attempt to use it 2024-08-15T17:46:12.578 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks 2024-08-15T17:46:12.578 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-15T17:46:12.580 INFO:teuthology.task.internal:Checking packages... 2024-08-15T17:46:12.601 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '2c991c196a651f8ef0f36c7763ec87a241b25549' 2024-08-15T17:46:12.601 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-15T17:46:12.601 INFO:teuthology.packaging:ref: None 2024-08-15T17:46:12.601 INFO:teuthology.packaging:tag: None 2024-08-15T17:46:12.601 INFO:teuthology.packaging:branch: wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-15T17:46:12.601 INFO:teuthology.packaging:sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T17:46:12.601 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-15T17:46:12.745 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1375-g2c991c19-1focal 2024-08-15T17:46:12.746 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-15T17:46:12.753 INFO:teuthology.task.internal:no buildpackages task found 2024-08-15T17:46:12.753 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-15T17:46:12.760 INFO:teuthology.task.internal:Saving configuration 2024-08-15T17:46:12.777 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-15T17:46:12.784 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-15T17:46:12.808 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-15 17:40:23.717788', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-08-15T17:46:12.829 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-15 17:40:23.716259', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-08-15T17:46:12.829 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-15T17:46:12.835 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-08-15T17:46:12.835 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-08-15T17:46:12.835 INFO:teuthology.run_tasks:Running task console_log... 2024-08-15T17:46:12.891 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb22e517ac0>, signals=[15]) 2024-08-15T17:46:12.891 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-15T17:46:12.896 INFO:teuthology.task.internal:Opening connections... 2024-08-15T17:46:12.896 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T17:46:12.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:46:12.994 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T17:46:12.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:46:13.091 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-15T17:46:13.096 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-08-15T17:46:13.103 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-08-15T17:46:13.104 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-08-15T17:46:13.150 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="Ubuntu" 2024-08-15T17:46:13.150 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-08-15T17:46:13.150 INFO:teuthology.orchestra.run.smithi038.stdout:ID=ubuntu 2024-08-15T17:46:13.150 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE=debian 2024-08-15T17:46:13.150 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-08-15T17:46:13.150 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="20.04" 2024-08-15T17:46:13.150 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-15T17:46:13.151 INFO:teuthology.orchestra.run.smithi038.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-15T17:46:13.151 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-15T17:46:13.151 INFO:teuthology.orchestra.run.smithi038.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-15T17:46:13.151 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_CODENAME=focal 2024-08-15T17:46:13.151 INFO:teuthology.orchestra.run.smithi038.stdout:UBUNTU_CODENAME=focal 2024-08-15T17:46:13.151 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-08-15T17:46:13.184 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-08-15T17:46:13.191 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-08-15T17:46:13.191 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-08-15T17:46:13.197 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="Ubuntu" 2024-08-15T17:46:13.197 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-08-15T17:46:13.197 INFO:teuthology.orchestra.run.smithi104.stdout:ID=ubuntu 2024-08-15T17:46:13.197 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE=debian 2024-08-15T17:46:13.197 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-08-15T17:46:13.197 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="20.04" 2024-08-15T17:46:13.197 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-15T17:46:13.197 INFO:teuthology.orchestra.run.smithi104.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-15T17:46:13.197 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-15T17:46:13.197 INFO:teuthology.orchestra.run.smithi104.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-15T17:46:13.198 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_CODENAME=focal 2024-08-15T17:46:13.198 INFO:teuthology.orchestra.run.smithi104.stdout:UBUNTU_CODENAME=focal 2024-08-15T17:46:13.198 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-08-15T17:46:13.218 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-15T17:46:13.227 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-15T17:46:13.298 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-15T17:46:13.298 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-08-15T17:46:13.301 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-08-15T17:46:13.306 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-15T17:46:13.312 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-15T17:46:13.312 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-08-15T17:46:13.350 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-08-15T17:46:13.373 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-15T17:46:13.424 INFO:teuthology.run_tasks:Running task kernel... 2024-08-15T17:46:13.438 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-08-15T17:46:13.438 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-08-15T17:46:13.439 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-15T17:46:13.439 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-15T17:46:13.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:46:13.445 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-08-15T17:46:13.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:46:13.446 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-08-15T17:46:13.453 INFO:teuthology.orchestra.run.smithi104.stdout:5.4.0-124-generic 2024-08-15T17:46:13.453 INFO:teuthology.task.kernel:Running kernel on smithi104: 5.4.0-124-generic 2024-08-15T17:46:13.453 DEBUG:teuthology.orchestra.run.smithi104:> sudo apt-get clean 2024-08-15T17:46:13.494 INFO:teuthology.orchestra.run.smithi038.stdout:5.4.0-124-generic 2024-08-15T17:46:13.495 INFO:teuthology.task.kernel:Running kernel on smithi038: 5.4.0-124-generic 2024-08-15T17:46:13.495 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-get clean 2024-08-15T17:46:13.613 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-get update 2024-08-15T17:46:13.648 DEBUG:teuthology.orchestra.run.smithi104:> sudo apt-get update 2024-08-15T17:46:13.775 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:46:13.776 INFO:teuthology.orchestra.run.smithi038.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-08-15T17:46:13.776 INFO:teuthology.orchestra.run.smithi038.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-08-15T17:46:13.851 INFO:teuthology.orchestra.run.smithi038.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-08-15T17:46:13.986 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:46:13.986 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-08-15T17:46:14.043 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-08-15T17:46:14.150 INFO:teuthology.orchestra.run.smithi038.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,125 kB] 2024-08-15T17:46:14.318 INFO:teuthology.orchestra.run.smithi038.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,016 kB] 2024-08-15T17:46:14.319 INFO:teuthology.orchestra.run.smithi038.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [796 kB] 2024-08-15T17:46:14.333 INFO:teuthology.orchestra.run.smithi038.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [464 kB] 2024-08-15T17:46:14.336 INFO:teuthology.orchestra.run.smithi038.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.1 kB] 2024-08-15T17:46:14.337 INFO:teuthology.orchestra.run.smithi038.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-08-15T17:46:14.337 INFO:teuthology.orchestra.run.smithi038.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,034 kB] 2024-08-15T17:46:14.359 INFO:teuthology.orchestra.run.smithi038.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [425 kB] 2024-08-15T17:46:14.362 INFO:teuthology.orchestra.run.smithi038.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [544 B] 2024-08-15T17:46:14.362 INFO:teuthology.orchestra.run.smithi038.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [999 kB] 2024-08-15T17:46:14.369 INFO:teuthology.orchestra.run.smithi038.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [674 kB] 2024-08-15T17:46:14.374 INFO:teuthology.orchestra.run.smithi038.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [212 kB] 2024-08-15T17:46:14.375 INFO:teuthology.orchestra.run.smithi038.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.1 kB] 2024-08-15T17:46:14.390 INFO:teuthology.orchestra.run.smithi038.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-08-15T17:46:14.391 INFO:teuthology.orchestra.run.smithi038.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-08-15T17:46:14.391 INFO:teuthology.orchestra.run.smithi038.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-08-15T17:46:14.392 INFO:teuthology.orchestra.run.smithi038.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-08-15T17:46:14.393 INFO:teuthology.orchestra.run.smithi038.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,495 kB] 2024-08-15T17:46:14.406 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-08-15T17:46:14.438 INFO:teuthology.orchestra.run.smithi038.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [543 kB] 2024-08-15T17:46:14.442 INFO:teuthology.orchestra.run.smithi038.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-08-15T17:46:14.442 INFO:teuthology.orchestra.run.smithi038.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,152 kB] 2024-08-15T17:46:14.463 INFO:teuthology.orchestra.run.smithi038.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-08-15T17:46:14.464 INFO:teuthology.orchestra.run.smithi038.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [442 kB] 2024-08-15T17:46:14.467 INFO:teuthology.orchestra.run.smithi038.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [540 B] 2024-08-15T17:46:14.467 INFO:teuthology.orchestra.run.smithi038.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [801 kB] 2024-08-15T17:46:14.472 INFO:teuthology.orchestra.run.smithi038.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,219 kB] 2024-08-15T17:46:14.481 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,125 kB] 2024-08-15T17:46:14.481 INFO:teuthology.orchestra.run.smithi038.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [294 kB] 2024-08-15T17:46:14.483 INFO:teuthology.orchestra.run.smithi038.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [27.8 kB] 2024-08-15T17:46:14.484 INFO:teuthology.orchestra.run.smithi038.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-08-15T17:46:14.484 INFO:teuthology.orchestra.run.smithi038.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-08-15T17:46:14.498 INFO:teuthology.orchestra.run.smithi038.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-08-15T17:46:14.499 INFO:teuthology.orchestra.run.smithi038.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-08-15T17:46:14.499 INFO:teuthology.orchestra.run.smithi038.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-08-15T17:46:14.499 INFO:teuthology.orchestra.run.smithi038.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-08-15T17:46:14.500 INFO:teuthology.orchestra.run.smithi038.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-08-15T17:46:14.517 INFO:teuthology.orchestra.run.smithi038.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-08-15T17:46:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-08-15T17:46:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-08-15T17:46:14.622 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,016 kB] 2024-08-15T17:46:14.970 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,495 kB] 2024-08-15T17:46:14.977 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [796 kB] 2024-08-15T17:46:14.991 INFO:teuthology.orchestra.run.smithi104.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [464 kB] 2024-08-15T17:46:15.000 INFO:teuthology.orchestra.run.smithi104.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.1 kB] 2024-08-15T17:46:15.000 INFO:teuthology.orchestra.run.smithi104.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-08-15T17:46:15.001 INFO:teuthology.orchestra.run.smithi104.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,034 kB] 2024-08-15T17:46:15.090 INFO:teuthology.orchestra.run.smithi104.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [425 kB] 2024-08-15T17:46:15.099 INFO:teuthology.orchestra.run.smithi104.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [544 B] 2024-08-15T17:46:15.099 INFO:teuthology.orchestra.run.smithi104.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [999 kB] 2024-08-15T17:46:15.147 INFO:teuthology.orchestra.run.smithi104.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [674 kB] 2024-08-15T17:46:15.161 INFO:teuthology.orchestra.run.smithi104.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [212 kB] 2024-08-15T17:46:15.167 INFO:teuthology.orchestra.run.smithi104.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.1 kB] 2024-08-15T17:46:15.168 INFO:teuthology.orchestra.run.smithi104.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-08-15T17:46:15.168 INFO:teuthology.orchestra.run.smithi104.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-08-15T17:46:15.168 INFO:teuthology.orchestra.run.smithi104.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-08-15T17:46:15.171 INFO:teuthology.orchestra.run.smithi104.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [543 kB] 2024-08-15T17:46:15.186 INFO:teuthology.orchestra.run.smithi104.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-08-15T17:46:15.187 INFO:teuthology.orchestra.run.smithi104.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,152 kB] 2024-08-15T17:46:15.244 INFO:teuthology.orchestra.run.smithi104.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-08-15T17:46:15.279 INFO:teuthology.orchestra.run.smithi104.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-08-15T17:46:15.304 INFO:teuthology.orchestra.run.smithi104.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [442 kB] 2024-08-15T17:46:15.311 INFO:teuthology.orchestra.run.smithi104.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [540 B] 2024-08-15T17:46:15.312 INFO:teuthology.orchestra.run.smithi104.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [801 kB] 2024-08-15T17:46:15.328 INFO:teuthology.orchestra.run.smithi104.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,219 kB] 2024-08-15T17:46:15.354 INFO:teuthology.orchestra.run.smithi104.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [294 kB] 2024-08-15T17:46:15.361 INFO:teuthology.orchestra.run.smithi104.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [27.8 kB] 2024-08-15T17:46:15.362 INFO:teuthology.orchestra.run.smithi104.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-08-15T17:46:15.388 INFO:teuthology.orchestra.run.smithi104.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-08-15T17:46:15.422 INFO:teuthology.orchestra.run.smithi104.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-08-15T17:46:15.422 INFO:teuthology.orchestra.run.smithi104.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-08-15T17:46:15.422 INFO:teuthology.orchestra.run.smithi104.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-08-15T17:46:15.423 INFO:teuthology.orchestra.run.smithi104.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-08-15T17:46:15.424 INFO:teuthology.orchestra.run.smithi104.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-08-15T17:46:15.505 INFO:teuthology.orchestra.run.smithi104.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-08-15T17:46:15.505 INFO:teuthology.orchestra.run.smithi104.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-08-15T17:46:15.506 INFO:teuthology.orchestra.run.smithi104.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-08-15T17:46:18.864 INFO:teuthology.orchestra.run.smithi038.stdout:Fetched 21.4 MB in 3s (6,844 kB/s) 2024-08-15T17:46:19.297 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 21.4 MB in 4s (6,078 kB/s) 2024-08-15T17:46:20.455 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2024-08-15T17:46:20.483 DEBUG:teuthology.orchestra.run.smithi038:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T17:46:20.555 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2024-08-15T17:46:20.749 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2024-08-15T17:46:20.755 INFO:teuthology.orchestra.run.smithi038.stdout:Building dependency tree... 2024-08-15T17:46:20.756 INFO:teuthology.orchestra.run.smithi038.stdout:Reading state information... 2024-08-15T17:46:20.776 DEBUG:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T17:46:20.869 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2024-08-15T17:46:20.960 INFO:teuthology.orchestra.run.smithi038.stdout:The following additional packages will be installed: 2024-08-15T17:46:20.961 INFO:teuthology.orchestra.run.smithi038.stdout: linux-generic linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T17:46:20.961 INFO:teuthology.orchestra.run.smithi038.stdout: linux-headers-generic linux-image-5.4.0-192-generic 2024-08-15T17:46:20.961 INFO:teuthology.orchestra.run.smithi038.stdout: linux-modules-5.4.0-192-generic linux-modules-extra-5.4.0-192-generic 2024-08-15T17:46:20.961 INFO:teuthology.orchestra.run.smithi038.stdout:Suggested packages: 2024-08-15T17:46:20.962 INFO:teuthology.orchestra.run.smithi038.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-08-15T17:46:20.962 INFO:teuthology.orchestra.run.smithi038.stdout:Recommended packages: 2024-08-15T17:46:20.962 INFO:teuthology.orchestra.run.smithi038.stdout: thermald 2024-08-15T17:46:21.029 INFO:teuthology.orchestra.run.smithi038.stdout:The following NEW packages will be installed: 2024-08-15T17:46:21.030 INFO:teuthology.orchestra.run.smithi038.stdout: linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T17:46:21.030 INFO:teuthology.orchestra.run.smithi038.stdout: linux-image-5.4.0-192-generic linux-modules-5.4.0-192-generic 2024-08-15T17:46:21.030 INFO:teuthology.orchestra.run.smithi038.stdout: linux-modules-extra-5.4.0-192-generic 2024-08-15T17:46:21.031 INFO:teuthology.orchestra.run.smithi038.stdout:The following packages will be upgraded: 2024-08-15T17:46:21.031 INFO:teuthology.orchestra.run.smithi038.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-15T17:46:21.092 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2024-08-15T17:46:21.092 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2024-08-15T17:46:21.217 INFO:teuthology.orchestra.run.smithi038.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T17:46:21.217 INFO:teuthology.orchestra.run.smithi038.stdout:Need to get 77.2 MB of archives. 2024-08-15T17:46:21.217 INFO:teuthology.orchestra.run.smithi038.stdout:After this operation, 381 MB of additional disk space will be used. 2024-08-15T17:46:21.217 INFO:teuthology.orchestra.run.smithi038.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-192-generic amd64 5.4.0-192.212 [15.0 MB] 2024-08-15T17:46:21.290 INFO:teuthology.orchestra.run.smithi104.stdout:The following additional packages will be installed: 2024-08-15T17:46:21.290 INFO:teuthology.orchestra.run.smithi104.stdout: linux-generic linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T17:46:21.290 INFO:teuthology.orchestra.run.smithi104.stdout: linux-headers-generic linux-image-5.4.0-192-generic 2024-08-15T17:46:21.290 INFO:teuthology.orchestra.run.smithi104.stdout: linux-modules-5.4.0-192-generic linux-modules-extra-5.4.0-192-generic 2024-08-15T17:46:21.291 INFO:teuthology.orchestra.run.smithi104.stdout:Suggested packages: 2024-08-15T17:46:21.291 INFO:teuthology.orchestra.run.smithi104.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-08-15T17:46:21.291 INFO:teuthology.orchestra.run.smithi104.stdout:Recommended packages: 2024-08-15T17:46:21.291 INFO:teuthology.orchestra.run.smithi104.stdout: thermald 2024-08-15T17:46:21.344 INFO:teuthology.orchestra.run.smithi104.stdout:The following NEW packages will be installed: 2024-08-15T17:46:21.344 INFO:teuthology.orchestra.run.smithi104.stdout: linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T17:46:21.345 INFO:teuthology.orchestra.run.smithi104.stdout: linux-image-5.4.0-192-generic linux-modules-5.4.0-192-generic 2024-08-15T17:46:21.345 INFO:teuthology.orchestra.run.smithi104.stdout: linux-modules-extra-5.4.0-192-generic 2024-08-15T17:46:21.346 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages will be upgraded: 2024-08-15T17:46:21.346 INFO:teuthology.orchestra.run.smithi104.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-15T17:46:21.401 INFO:teuthology.orchestra.run.smithi104.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T17:46:21.401 INFO:teuthology.orchestra.run.smithi104.stdout:Need to get 77.2 MB of archives. 2024-08-15T17:46:21.402 INFO:teuthology.orchestra.run.smithi104.stdout:After this operation, 381 MB of additional disk space will be used. 2024-08-15T17:46:21.402 INFO:teuthology.orchestra.run.smithi104.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-192-generic amd64 5.4.0-192.212 [15.0 MB] 2024-08-15T17:46:21.749 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-192-generic amd64 5.4.0-192.212 [10.5 MB] 2024-08-15T17:46:21.840 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-192-generic amd64 5.4.0-192.212 [39.2 MB] 2024-08-15T17:46:22.309 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.192.190 [1,900 B] 2024-08-15T17:46:22.309 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.192.190 [2,588 B] 2024-08-15T17:46:22.309 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192 all 5.4.0-192.212 [11.0 MB] 2024-08-15T17:46:22.403 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192-generic amd64 5.4.0-192.212 [1,398 kB] 2024-08-15T17:46:22.404 INFO:teuthology.orchestra.run.smithi038.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-192-generic amd64 5.4.0-192.212 [10.5 MB] 2024-08-15T17:46:22.414 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.192.190 [2,456 B] 2024-08-15T17:46:22.730 INFO:teuthology.orchestra.run.smithi038.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-192-generic amd64 5.4.0-192.212 [39.2 MB] 2024-08-15T17:46:23.111 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 77.2 MB in 1s (72.8 MB/s) 2024-08-15T17:46:23.315 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-modules-5.4.0-192-generic. 2024-08-15T17:46:23.999 INFO:teuthology.orchestra.run.smithi038.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.192.190 [1,900 B] 2024-08-15T17:46:23.999 INFO:teuthology.orchestra.run.smithi038.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.192.190 [2,588 B] 2024-08-15T17:46:23.999 INFO:teuthology.orchestra.run.smithi038.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192 all 5.4.0-192.212 [11.0 MB] 2024-08-15T17:46:24.277 INFO:teuthology.orchestra.run.smithi038.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192-generic amd64 5.4.0-192.212 [1,398 kB] 2024-08-15T17:46:24.293 INFO:teuthology.orchestra.run.smithi038.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.192.190 [2,456 B] 2024-08-15T17:46:25.033 INFO:teuthology.orchestra.run.smithi038.stdout:Fetched 77.2 MB in 3s (23.7 MB/s) 2024-08-15T17:46:25.090 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-08-15T17:46:25.095 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../0-linux-modules-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:46:25.223 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:25.269 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package linux-modules-5.4.0-192-generic. 2024-08-15T17:46:26.973 INFO:teuthology.orchestra.run.smithi038.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-08-15T17:46:26.978 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../0-linux-modules-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:46:27.100 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:27.247 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-image-5.4.0-192-generic. 2024-08-15T17:46:27.263 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../1-linux-image-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:46:27.358 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:28.491 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-192-generic. 2024-08-15T17:46:28.505 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:46:28.549 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:29.000 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package linux-image-5.4.0-192-generic. 2024-08-15T17:46:29.014 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../1-linux-image-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:46:29.139 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:30.229 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-192-generic. 2024-08-15T17:46:30.242 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:46:30.279 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:33.468 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../3-linux-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:46:33.595 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:46:33.957 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:46:34.107 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-image-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:46:34.435 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-headers-5.4.0-192. 2024-08-15T17:46:34.450 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../5-linux-headers-5.4.0-192_5.4.0-192.212_all.deb ... 2024-08-15T17:46:34.493 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T17:46:35.153 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../3-linux-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:46:35.294 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:46:35.662 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:46:35.789 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-image-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:46:36.142 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package linux-headers-5.4.0-192. 2024-08-15T17:46:36.155 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../5-linux-headers-5.4.0-192_5.4.0-192.212_all.deb ... 2024-08-15T17:46:36.183 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T17:46:39.995 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-headers-5.4.0-192-generic. 2024-08-15T17:46:40.016 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../6-linux-headers-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:46:40.053 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:42.372 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package linux-headers-5.4.0-192-generic. 2024-08-15T17:46:42.394 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../6-linux-headers-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:46:42.447 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:42.806 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:46:42.949 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-headers-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:46:43.291 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T17:46:43.429 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:44.638 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:45.281 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:46:45.417 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-headers-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:46:45.787 INFO:teuthology.orchestra.run.smithi104.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:46:45.787 INFO:teuthology.orchestra.run.smithi104.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-192-generic 2024-08-15T17:46:45.810 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T17:46:45.931 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:45.963 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:46.033 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:47.225 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-image-generic (5.4.0.192.190) ... 2024-08-15T17:46:47.273 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:47.351 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-headers-generic (5.4.0.192.190) ... 2024-08-15T17:46:47.502 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-generic (5.4.0.192.190) ... 2024-08-15T17:46:47.637 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:47.711 INFO:teuthology.orchestra.run.smithi104.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-15T17:46:47.711 INFO:teuthology.orchestra.run.smithi104.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:46:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:46:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-192-generic 2024-08-15T17:46:49.059 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:49.202 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:50.421 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-image-generic (5.4.0.192.190) ... 2024-08-15T17:46:50.571 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-headers-generic (5.4.0.192.190) ... 2024-08-15T17:46:50.706 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-generic (5.4.0.192.190) ... 2024-08-15T17:46:50.832 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:46:50.921 INFO:teuthology.orchestra.run.smithi038.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-15T17:46:50.922 INFO:teuthology.orchestra.run.smithi038.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:47:09.466 INFO:teuthology.orchestra.run.smithi104.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-15T17:47:09.467 INFO:teuthology.orchestra.run.smithi104.stdout:Sourcing file `/etc/default/grub' 2024-08-15T17:47:09.472 INFO:teuthology.orchestra.run.smithi104.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T17:47:09.492 INFO:teuthology.orchestra.run.smithi104.stdout:Generating grub configuration file ... 2024-08-15T17:47:09.860 INFO:teuthology.orchestra.run.smithi104.stdout:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:47:09.890 INFO:teuthology.orchestra.run.smithi104.stdout:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:47:10.274 INFO:teuthology.orchestra.run.smithi104.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T17:47:10.285 INFO:teuthology.orchestra.run.smithi104.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T17:47:13.129 INFO:teuthology.orchestra.run.smithi104.stdout:done 2024-08-15T17:47:13.473 INFO:teuthology.orchestra.run.smithi038.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-15T17:47:13.473 INFO:teuthology.orchestra.run.smithi038.stdout:Sourcing file `/etc/default/grub' 2024-08-15T17:47:13.502 INFO:teuthology.orchestra.run.smithi038.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T17:47:13.568 INFO:teuthology.orchestra.run.smithi038.stdout:Generating grub configuration file ... 2024-08-15T17:47:13.571 DEBUG:teuthology.orchestra.run.smithi104:> dpkg -s linux-image-generic 2024-08-15T17:47:13.612 INFO:teuthology.orchestra.run.smithi104.stdout:Package: linux-image-generic 2024-08-15T17:47:13.613 INFO:teuthology.orchestra.run.smithi104.stdout:Status: install ok installed 2024-08-15T17:47:13.613 INFO:teuthology.orchestra.run.smithi104.stdout:Priority: optional 2024-08-15T17:47:13.613 INFO:teuthology.orchestra.run.smithi104.stdout:Section: kernel 2024-08-15T17:47:13.613 INFO:teuthology.orchestra.run.smithi104.stdout:Installed-Size: 20 2024-08-15T17:47:13.613 INFO:teuthology.orchestra.run.smithi104.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T17:47:13.613 INFO:teuthology.orchestra.run.smithi104.stdout:Architecture: amd64 2024-08-15T17:47:13.613 INFO:teuthology.orchestra.run.smithi104.stdout:Source: linux-meta 2024-08-15T17:47:13.613 INFO:teuthology.orchestra.run.smithi104.stdout:Version: 5.4.0.192.190 2024-08-15T17:47:13.613 INFO:teuthology.orchestra.run.smithi104.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T17:47:13.614 INFO:teuthology.orchestra.run.smithi104.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T17:47:13.614 INFO:teuthology.orchestra.run.smithi104.stdout:Recommends: thermald 2024-08-15T17:47:13.614 INFO:teuthology.orchestra.run.smithi104.stdout:Description: Generic Linux kernel image 2024-08-15T17:47:13.614 INFO:teuthology.orchestra.run.smithi104.stdout: This package will always depend on the latest generic kernel image 2024-08-15T17:47:13.614 INFO:teuthology.orchestra.run.smithi104.stdout: available. 2024-08-15T17:47:13.614 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-192-generic 2024-08-15T17:47:13.614 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-15T17:47:13.614 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-15T17:47:13.614 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-08-15T17:47:13.615 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi104.front.sepia.ceph.com, path=None, version=distro) 2024-08-15T17:47:13.615 DEBUG:teuthology.orchestra.run.smithi104:> sudo apt-get clean 2024-08-15T17:47:13.704 DEBUG:teuthology.orchestra.run.smithi104:> sudo apt-get update 2024-08-15T17:47:13.839 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:47:13.839 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T17:47:13.845 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T17:47:13.862 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T17:47:13.912 INFO:teuthology.orchestra.run.smithi038.stdout:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:47:13.947 INFO:teuthology.orchestra.run.smithi038.stdout:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:47:14.433 INFO:teuthology.orchestra.run.smithi038.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T17:47:14.445 INFO:teuthology.orchestra.run.smithi038.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T17:47:15.686 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2024-08-15T17:47:15.713 DEBUG:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T17:47:15.783 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2024-08-15T17:47:15.991 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2024-08-15T17:47:15.992 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2024-08-15T17:47:16.218 INFO:teuthology.orchestra.run.smithi104.stdout:linux-image-generic is already the newest version (5.4.0.192.190). 2024-08-15T17:47:16.218 INFO:teuthology.orchestra.run.smithi104.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T17:47:16.220 DEBUG:teuthology.orchestra.run.smithi104:> dpkg -s linux-image-generic 2024-08-15T17:47:16.239 INFO:teuthology.orchestra.run.smithi104.stdout:Package: linux-image-generic 2024-08-15T17:47:16.239 INFO:teuthology.orchestra.run.smithi104.stdout:Status: install ok installed 2024-08-15T17:47:16.239 INFO:teuthology.orchestra.run.smithi104.stdout:Priority: optional 2024-08-15T17:47:16.239 INFO:teuthology.orchestra.run.smithi104.stdout:Section: kernel 2024-08-15T17:47:16.239 INFO:teuthology.orchestra.run.smithi104.stdout:Installed-Size: 20 2024-08-15T17:47:16.239 INFO:teuthology.orchestra.run.smithi104.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T17:47:16.239 INFO:teuthology.orchestra.run.smithi104.stdout:Architecture: amd64 2024-08-15T17:47:16.239 INFO:teuthology.orchestra.run.smithi104.stdout:Source: linux-meta 2024-08-15T17:47:16.239 INFO:teuthology.orchestra.run.smithi104.stdout:Version: 5.4.0.192.190 2024-08-15T17:47:16.239 INFO:teuthology.orchestra.run.smithi104.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T17:47:16.240 INFO:teuthology.orchestra.run.smithi104.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T17:47:16.240 INFO:teuthology.orchestra.run.smithi104.stdout:Recommends: thermald 2024-08-15T17:47:16.240 INFO:teuthology.orchestra.run.smithi104.stdout:Description: Generic Linux kernel image 2024-08-15T17:47:16.240 INFO:teuthology.orchestra.run.smithi104.stdout: This package will always depend on the latest generic kernel image 2024-08-15T17:47:16.240 INFO:teuthology.orchestra.run.smithi104.stdout: available. 2024-08-15T17:47:16.240 DEBUG:teuthology.orchestra.run.smithi104:> mktemp 2024-08-15T17:47:16.246 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.0rML6TkhHz 2024-08-15T17:47:16.246 DEBUG:teuthology.orchestra.run.smithi104:> sudo cp /boot/grub/grub.cfg /tmp/tmp.0rML6TkhHz 2024-08-15T17:47:16.330 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.0rML6TkhHz 2024-08-15T17:47:16.377 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.0rML6TkhHz is 9KB 2024-08-15T17:47:16.391 DEBUG:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.0rML6TkhHz 2024-08-15T17:47:16.397 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-15T17:47:16.453 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:47:16.453 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-15T17:47:16.453 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-15T17:47:16.482 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-192-generic 2024-08-15T17:47:16.482 DEBUG:teuthology.orchestra.run.smithi104:> sudo update-grub 2024-08-15T17:47:17.183 INFO:teuthology.orchestra.run.smithi038.stdout:done 2024-08-15T17:47:17.608 DEBUG:teuthology.orchestra.run.smithi038:> dpkg -s linux-image-generic 2024-08-15T17:47:17.651 INFO:teuthology.orchestra.run.smithi038.stdout:Package: linux-image-generic 2024-08-15T17:47:17.651 INFO:teuthology.orchestra.run.smithi038.stdout:Status: install ok installed 2024-08-15T17:47:17.651 INFO:teuthology.orchestra.run.smithi038.stdout:Priority: optional 2024-08-15T17:47:17.651 INFO:teuthology.orchestra.run.smithi038.stdout:Section: kernel 2024-08-15T17:47:17.651 INFO:teuthology.orchestra.run.smithi038.stdout:Installed-Size: 20 2024-08-15T17:47:17.651 INFO:teuthology.orchestra.run.smithi038.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T17:47:17.651 INFO:teuthology.orchestra.run.smithi038.stdout:Architecture: amd64 2024-08-15T17:47:17.651 INFO:teuthology.orchestra.run.smithi038.stdout:Source: linux-meta 2024-08-15T17:47:17.651 INFO:teuthology.orchestra.run.smithi038.stdout:Version: 5.4.0.192.190 2024-08-15T17:47:17.651 INFO:teuthology.orchestra.run.smithi038.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T17:47:17.651 INFO:teuthology.orchestra.run.smithi038.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T17:47:17.652 INFO:teuthology.orchestra.run.smithi038.stdout:Recommends: thermald 2024-08-15T17:47:17.652 INFO:teuthology.orchestra.run.smithi038.stdout:Description: Generic Linux kernel image 2024-08-15T17:47:17.652 INFO:teuthology.orchestra.run.smithi038.stdout: This package will always depend on the latest generic kernel image 2024-08-15T17:47:17.652 INFO:teuthology.orchestra.run.smithi038.stdout: available. 2024-08-15T17:47:17.652 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-192-generic 2024-08-15T17:47:17.652 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-15T17:47:17.652 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-15T17:47:17.652 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-08-15T17:47:17.652 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi038.front.sepia.ceph.com, path=None, version=distro) 2024-08-15T17:47:17.653 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-get clean 2024-08-15T17:47:17.743 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-get update 2024-08-15T17:47:17.866 INFO:teuthology.orchestra.run.smithi104.stderr:Sourcing file `/etc/default/grub' 2024-08-15T17:47:17.871 INFO:teuthology.orchestra.run.smithi104.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T17:47:17.878 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:47:17.881 INFO:teuthology.orchestra.run.smithi104.stderr:Generating grub configuration file ... 2024-08-15T17:47:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T17:47:17.902 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T17:47:17.996 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T17:47:18.149 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:47:18.160 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:47:18.522 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T17:47:18.534 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T17:47:19.814 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2024-08-15T17:47:19.841 DEBUG:teuthology.orchestra.run.smithi038:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T17:47:19.915 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2024-08-15T17:47:20.113 INFO:teuthology.orchestra.run.smithi038.stdout:Building dependency tree... 2024-08-15T17:47:20.114 INFO:teuthology.orchestra.run.smithi038.stdout:Reading state information... 2024-08-15T17:47:20.177 INFO:teuthology.orchestra.run.smithi104.stderr:done 2024-08-15T17:47:20.180 DEBUG:teuthology.orchestra.run.smithi104:> sudo shutdown -r now 2024-08-15T17:47:20.353 INFO:teuthology.orchestra.run.smithi038.stdout:linux-image-generic is already the newest version (5.4.0.192.190). 2024-08-15T17:47:20.353 INFO:teuthology.orchestra.run.smithi038.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T17:47:20.355 DEBUG:teuthology.orchestra.run.smithi038:> dpkg -s linux-image-generic 2024-08-15T17:47:20.376 INFO:teuthology.orchestra.run.smithi038.stdout:Package: linux-image-generic 2024-08-15T17:47:20.376 INFO:teuthology.orchestra.run.smithi038.stdout:Status: install ok installed 2024-08-15T17:47:20.376 INFO:teuthology.orchestra.run.smithi038.stdout:Priority: optional 2024-08-15T17:47:20.376 INFO:teuthology.orchestra.run.smithi038.stdout:Section: kernel 2024-08-15T17:47:20.377 INFO:teuthology.orchestra.run.smithi038.stdout:Installed-Size: 20 2024-08-15T17:47:20.377 INFO:teuthology.orchestra.run.smithi038.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T17:47:20.377 INFO:teuthology.orchestra.run.smithi038.stdout:Architecture: amd64 2024-08-15T17:47:20.377 INFO:teuthology.orchestra.run.smithi038.stdout:Source: linux-meta 2024-08-15T17:47:20.377 INFO:teuthology.orchestra.run.smithi038.stdout:Version: 5.4.0.192.190 2024-08-15T17:47:20.377 INFO:teuthology.orchestra.run.smithi038.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T17:47:20.377 INFO:teuthology.orchestra.run.smithi038.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T17:47:20.377 INFO:teuthology.orchestra.run.smithi038.stdout:Recommends: thermald 2024-08-15T17:47:20.377 INFO:teuthology.orchestra.run.smithi038.stdout:Description: Generic Linux kernel image 2024-08-15T17:47:20.377 INFO:teuthology.orchestra.run.smithi038.stdout: This package will always depend on the latest generic kernel image 2024-08-15T17:47:20.377 INFO:teuthology.orchestra.run.smithi038.stdout: available. 2024-08-15T17:47:20.378 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2024-08-15T17:47:20.382 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.I8tRJW2sW6 2024-08-15T17:47:20.383 DEBUG:teuthology.orchestra.run.smithi038:> sudo cp /boot/grub/grub.cfg /tmp/tmp.I8tRJW2sW6 2024-08-15T17:47:20.440 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.I8tRJW2sW6 2024-08-15T17:47:20.474 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.I8tRJW2sW6 is 9KB 2024-08-15T17:47:20.487 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.I8tRJW2sW6 2024-08-15T17:47:20.493 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-15T17:47:20.549 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:47:20.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-15T17:47:20.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-15T17:47:20.581 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-192-generic 2024-08-15T17:47:20.581 DEBUG:teuthology.orchestra.run.smithi038:> sudo update-grub 2024-08-15T17:47:22.058 INFO:teuthology.orchestra.run.smithi038.stderr:Sourcing file `/etc/default/grub' 2024-08-15T17:47:22.076 INFO:teuthology.orchestra.run.smithi038.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T17:47:22.084 INFO:teuthology.orchestra.run.smithi038.stderr:Generating grub configuration file ... 2024-08-15T17:47:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:47:22.376 INFO:teuthology.orchestra.run.smithi038.stderr:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:47:22.806 INFO:teuthology.orchestra.run.smithi038.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T17:47:22.818 INFO:teuthology.orchestra.run.smithi038.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T17:47:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:done 2024-08-15T17:47:24.500 DEBUG:teuthology.orchestra.run.smithi038:> sudo shutdown -r now 2024-08-15T17:47:50.208 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-08-15T17:47:50.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-08-15T17:47:50.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:47:54.508 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-08-15T17:47:54.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-08-15T17:47:54.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:48:08.542 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-08-15T17:48:17.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-08-15T17:48:17.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:48:20.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-08-15T17:48:32.618 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-08-15T17:48:32.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:48:51.043 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-08-15T17:48:54.514 DEBUG:teuthology.orchestra.remote:timed out 2024-08-15T17:49:03.523 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-08-15T17:49:03.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:49:06.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-08-15T17:49:06.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:49:06.294 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-08-15T17:49:06.921 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-08-15T17:49:06.921 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-192-generic"... 2024-08-15T17:49:06.922 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-08-15T17:49:06.970 INFO:teuthology.orchestra.run.smithi104.stdout:5.4.0-192-generic 2024-08-15T17:49:06.971 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-192-generic vs 5.4.0-192-generic 2024-08-15T17:49:06.971 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-15T17:49:06.971 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-15T17:49:06.981 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-08-15T17:49:07.972 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-08-15T17:49:07.972 DEBUG:teuthology.orchestra.run.smithi104:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-15T17:49:08.071 INFO:teuthology.orchestra.run.smithi104.stdout:ttyS1 2024-08-15T17:49:08.095 DEBUG:teuthology.parallel:result is None 2024-08-15T17:49:08.096 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-08-15T17:49:08.096 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-192-generic"... 2024-08-15T17:49:08.096 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-08-15T17:49:08.103 INFO:teuthology.orchestra.run.smithi038.stdout:5.4.0-192-generic 2024-08-15T17:49:08.103 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-192-generic vs 5.4.0-192-generic 2024-08-15T17:49:08.103 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-15T17:49:08.103 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-15T17:49:09.103 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-08-15T17:49:09.104 DEBUG:teuthology.orchestra.run.smithi038:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-15T17:49:09.219 INFO:teuthology.orchestra.run.smithi038.stdout:ttyS1 2024-08-15T17:49:09.238 DEBUG:teuthology.parallel:result is None 2024-08-15T17:49:09.238 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-15T17:49:09.245 INFO:teuthology.task.internal:Creating test directory... 2024-08-15T17:49:09.245 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-15T17:49:09.247 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-15T17:49:09.254 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-15T17:49:09.260 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-15T17:49:09.266 INFO:teuthology.task.internal:Creating archive directory... 2024-08-15T17:49:09.266 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-15T17:49:09.268 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-15T17:49:09.320 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-15T17:49:09.327 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-15T17:49:09.327 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-15T17:49:09.330 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-15T17:49:09.347 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T17:49:09.354 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T17:49:09.358 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T17:49:09.366 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T17:49:09.367 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-15T17:49:09.374 INFO:teuthology.task.internal:Configuring sudo... 2024-08-15T17:49:09.374 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-15T17:49:09.376 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-15T17:49:09.393 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-15T17:49:09.403 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-15T17:49:09.403 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-15T17:49:09.433 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-15T17:49:09.443 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-15T17:49:09.449 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-15T17:49:09.456 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:49:09.456 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-15T17:49:09.533 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-15T17:49:09.541 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-15T17:49:09.591 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:49:09.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-15T17:49:09.616 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-08-15T17:49:09.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-08-15T17:49:09.731 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-15T17:49:09.738 INFO:teuthology.task.internal:Starting timer... 2024-08-15T17:49:09.738 INFO:teuthology.run_tasks:Running task pcp... 2024-08-15T17:49:09.750 INFO:teuthology.run_tasks:Running task selinux... 2024-08-15T17:49:09.759 DEBUG:teuthology.task.selinux:Excluding smithi038: OS 'ubuntu' does not support SELinux 2024-08-15T17:49:09.759 DEBUG:teuthology.task.selinux:Excluding smithi104: OS 'ubuntu' does not support SELinux 2024-08-15T17:49:09.759 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-15T17:49:09.759 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-08-15T17:49:09.759 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-15T17:49:09.759 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-15T17:49:09.767 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-15T17:49:09.778 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-15T17:49:09.779 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi038.front.sepia.ceph.com,smithi104.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-15T17:54:46.881 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com')] 2024-08-15T17:54:46.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-08-15T17:54:46.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:54:46.983 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-08-15T17:54:47.112 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-08-15T17:54:47.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-08-15T17:54:47.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:54:47.215 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-08-15T17:54:47.340 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-08-15T17:54:47.340 INFO:teuthology.run_tasks:Running task clock... 2024-08-15T17:54:47.351 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-15T17:54:47.352 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-15T17:54:47.352 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T17:54:47.354 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-15T17:54:47.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T17:54:47.390 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-08-15T17:54:47.390 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: Command line: ntpd -gq 2024-08-15T17:54:47.391 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: proto: precision = 0.103 usec (-23) 2024-08-15T17:54:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T17:54:47.392 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T17:54:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-15T17:54:47.393 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: restrict ::: KOD does nothing without LIMITED. 2024-08-15T17:54:47.394 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: Listen and drop on 0 v6wildcard [::]:123 2024-08-15T17:54:47.394 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-15T17:54:47.394 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: Listen normally on 2 lo 127.0.0.1:123 2024-08-15T17:54:47.394 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: Listen normally on 3 ens1f0 172.21.15.38:123 2024-08-15T17:54:47.394 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: Listen normally on 4 lo [::1]:123 2024-08-15T17:54:47.394 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15fc%3]:123 2024-08-15T17:54:47.394 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:47 ntpd[13472]: Listening on routing socket on fd #22 for interface updates 2024-08-15T17:54:47.419 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-08-15T17:54:47.419 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: Command line: ntpd -gq 2024-08-15T17:54:47.420 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: proto: precision = 0.087 usec (-23) 2024-08-15T17:54:47.420 INFO:teuthology.orchestra.run.smithi104.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T17:54:47.421 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T17:54:47.421 INFO:teuthology.orchestra.run.smithi104.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-15T17:54:47.422 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: restrict ::: KOD does nothing without LIMITED. 2024-08-15T17:54:47.422 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: Listen and drop on 0 v6wildcard [::]:123 2024-08-15T17:54:47.422 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-15T17:54:47.422 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: Listen normally on 2 lo 127.0.0.1:123 2024-08-15T17:54:47.423 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: Listen normally on 3 enp3s0f1 172.21.15.104:123 2024-08-15T17:54:47.423 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: Listen normally on 4 lo [::1]:123 2024-08-15T17:54:47.423 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce89%5]:123 2024-08-15T17:54:47.423 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:47 ntpd[13605]: Listening on routing socket on fd #22 for interface updates 2024-08-15T17:54:48.391 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:48 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:48.391 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:48 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:48.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:48 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:48.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:48 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:48.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:48 ntpd[13472]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:48.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:48 ntpd[13472]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:48.419 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:48 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:48.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:48 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:49.391 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:49 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:49.391 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:49 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:50.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:50.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:50.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:50.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:50.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:50.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:50.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:50.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:50.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:50.394 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:50.394 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:50.394 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:50.394 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:50.394 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:50.395 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:50.395 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:50.395 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:50.395 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:50 ntpd[13472]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:51.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:51 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:51.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:51 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:51.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:51 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:51.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:51 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:51.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:51 ntpd[13605]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:51.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:51 ntpd[13605]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:52.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:52.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:52.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:52.394 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:52.394 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:52 ntpd[13472]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:52 ntpd[13605]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:53.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:53 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:53.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:53 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:53.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:53 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:53.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:53 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:53.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:53 ntpd[13605]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:53.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:53 ntpd[13605]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:54.391 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:54.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:54.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:54.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:54.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:54.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:54.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:54.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:54.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:54.392 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:54.393 INFO:teuthology.orchestra.run.smithi038.stdout:15 Aug 17:54:54 ntpd[13472]: ntpd: time slew +0.000744 s 2024-08-15T17:54:54.393 INFO:teuthology.orchestra.run.smithi038.stdout:ntpd: time slew +0.000744s 2024-08-15T17:54:54.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-15T17:54:54.393 INFO:teuthology.orchestra.run.smithi038.stderr:15 Aug 17:54:54 ntpd[13472]: can't open /var/log/ntpstats/loopstats.20240815: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:54.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:54 ntpd[13605]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T17:54:54.457 INFO:teuthology.orchestra.run.smithi038.stdout:============================================================================== 2024-08-15T17:54:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:54:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:54:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:54:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:54:55.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:55 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:55.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:55 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:55.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:55 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:55.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:55 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:55.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:55 ntpd[13605]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:55.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:55 ntpd[13605]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:56.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:56.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:56.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:56.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:56.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:56.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:56.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:56.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:56.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:56.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:56.422 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:56.422 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:56 ntpd[13605]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:57.420 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:57 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:57.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:57 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:57.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:57 ntpd[13605]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:54:57.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:57 ntpd[13605]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:54:57.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:57 ntpd[13605]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:54:57.421 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:57 ntpd[13605]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:54:57.421 INFO:teuthology.orchestra.run.smithi104.stdout:15 Aug 17:54:57 ntpd[13605]: ntpd: time slew +0.004670 s 2024-08-15T17:54:57.421 INFO:teuthology.orchestra.run.smithi104.stdout:ntpd: time slew +0.004670s 2024-08-15T17:54:57.422 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:57 ntpd[13605]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-15T17:54:57.422 INFO:teuthology.orchestra.run.smithi104.stderr:15 Aug 17:54:57 ntpd[13605]: can't open /var/log/ntpstats/loopstats.20240815: Permission denied 2024-08-15T17:54:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T17:54:57.486 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================== 2024-08-15T17:54:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:54:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:54:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:54:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:54:57.487 INFO:teuthology.run_tasks:Running task install... 2024-08-15T17:54:57.496 DEBUG:teuthology.task.install:project ceph 2024-08-15T17:54:57.496 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}} 2024-08-15T17:54:57.496 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'} 2024-08-15T17:54:57.497 INFO:teuthology.task.install:Using flavor: default 2024-08-15T17:54:57.502 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-15T17:54:57.502 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-08-15T17:54:57.502 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-key list | grep Ceph 2024-08-15T17:54:57.503 DEBUG:teuthology.orchestra.run.smithi104:> sudo apt-key list | grep Ceph 2024-08-15T17:54:57.605 INFO:teuthology.orchestra.run.smithi104.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-08-15T17:54:57.610 INFO:teuthology.orchestra.run.smithi038.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-08-15T17:54:57.665 INFO:teuthology.orchestra.run.smithi104.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-15T17:54:57.666 INFO:teuthology.orchestra.run.smithi104.stdout:uid [ unknown] Ceph.com (release key) 2024-08-15T17:54:57.666 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-08-15T17:54:57.666 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-08-15T17:54:57.667 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T17:54:57.671 INFO:teuthology.orchestra.run.smithi038.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-15T17:54:57.671 INFO:teuthology.orchestra.run.smithi038.stdout:uid [ unknown] Ceph.com (release key) 2024-08-15T17:54:57.672 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-08-15T17:54:57.672 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-08-15T17:54:57.672 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T17:54:57.813 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default/ 2024-08-15T17:54:57.814 INFO:teuthology.task.install.deb:Package version is 17.2.7-1375-g2c991c19-1focal 2024-08-15T17:54:57.823 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default/ 2024-08-15T17:54:57.823 INFO:teuthology.task.install.deb:Package version is 17.2.7-1375-g2c991c19-1focal 2024-08-15T17:54:57.924 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:54:57.924 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-15T17:54:57.932 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:54:57.933 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-15T17:54:57.941 DEBUG:teuthology.orchestra.run.smithi104:> sudo apt-get update 2024-08-15T17:54:57.950 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-get update 2024-08-15T17:54:58.094 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:54:58.094 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T17:54:58.100 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T17:54:58.118 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T17:54:58.127 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T17:54:58.151 INFO:teuthology.orchestra.run.smithi038.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal InRelease 2024-08-15T17:54:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-08-15T17:54:58.183 INFO:teuthology.orchestra.run.smithi104.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal InRelease 2024-08-15T17:54:58.196 INFO:teuthology.orchestra.run.smithi038.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release.gpg 2024-08-15T17:54:58.205 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-08-15T17:54:58.217 INFO:teuthology.orchestra.run.smithi038.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-08-15T17:54:58.228 INFO:teuthology.orchestra.run.smithi104.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release.gpg 2024-08-15T17:54:58.244 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:54:58.250 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-08-15T17:54:58.250 INFO:teuthology.orchestra.run.smithi038.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main i386 Packages [3,369 B] 2024-08-15T17:54:58.290 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main i386 Packages [3,369 B] 2024-08-15T17:54:58.329 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T17:54:58.411 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T17:54:58.767 INFO:teuthology.orchestra.run.smithi038.stdout:Fetched 28.4 kB in 1s (44.0 kB/s) 2024-08-15T17:54:58.803 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 28.4 kB in 1s (44.0 kB/s) 2024-08-15T17:54:59.949 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2024-08-15T17:54:59.953 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2024-08-15T17:54:59.976 DEBUG:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1375-g2c991c19-1focal cephadm=17.2.7-1375-g2c991c19-1focal ceph-mds=17.2.7-1375-g2c991c19-1focal ceph-mgr=17.2.7-1375-g2c991c19-1focal ceph-common=17.2.7-1375-g2c991c19-1focal ceph-fuse=17.2.7-1375-g2c991c19-1focal ceph-test=17.2.7-1375-g2c991c19-1focal ceph-volume=17.2.7-1375-g2c991c19-1focal radosgw=17.2.7-1375-g2c991c19-1focal python3-rados=17.2.7-1375-g2c991c19-1focal python3-rgw=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal python3-rbd=17.2.7-1375-g2c991c19-1focal libcephfs2=17.2.7-1375-g2c991c19-1focal libcephfs-dev=17.2.7-1375-g2c991c19-1focal librados2=17.2.7-1375-g2c991c19-1focal librbd1=17.2.7-1375-g2c991c19-1focal rbd-fuse=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal cephfs-shell=17.2.7-1375-g2c991c19-1focal cephfs-top=17.2.7-1375-g2c991c19-1focal cephfs-mirror=17.2.7-1375-g2c991c19-1focal 2024-08-15T17:54:59.980 DEBUG:teuthology.orchestra.run.smithi038:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1375-g2c991c19-1focal cephadm=17.2.7-1375-g2c991c19-1focal ceph-mds=17.2.7-1375-g2c991c19-1focal ceph-mgr=17.2.7-1375-g2c991c19-1focal ceph-common=17.2.7-1375-g2c991c19-1focal ceph-fuse=17.2.7-1375-g2c991c19-1focal ceph-test=17.2.7-1375-g2c991c19-1focal ceph-volume=17.2.7-1375-g2c991c19-1focal radosgw=17.2.7-1375-g2c991c19-1focal python3-rados=17.2.7-1375-g2c991c19-1focal python3-rgw=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal python3-rbd=17.2.7-1375-g2c991c19-1focal libcephfs2=17.2.7-1375-g2c991c19-1focal libcephfs-dev=17.2.7-1375-g2c991c19-1focal librados2=17.2.7-1375-g2c991c19-1focal librbd1=17.2.7-1375-g2c991c19-1focal rbd-fuse=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal cephfs-shell=17.2.7-1375-g2c991c19-1focal cephfs-top=17.2.7-1375-g2c991c19-1focal cephfs-mirror=17.2.7-1375-g2c991c19-1focal 2024-08-15T17:55:00.043 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2024-08-15T17:55:00.049 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2024-08-15T17:55:00.256 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2024-08-15T17:55:00.256 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2024-08-15T17:55:00.270 INFO:teuthology.orchestra.run.smithi038.stdout:Building dependency tree... 2024-08-15T17:55:00.271 INFO:teuthology.orchestra.run.smithi038.stdout:Reading state information... 2024-08-15T17:55:00.448 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T17:55:00.453 INFO:teuthology.orchestra.run.smithi104.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T17:55:00.453 INFO:teuthology.orchestra.run.smithi104.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T17:55:00.453 INFO:teuthology.orchestra.run.smithi104.stdout:The following additional packages will be installed: 2024-08-15T17:55:00.453 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-15T17:55:00.453 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-08-15T17:55:00.453 INFO:teuthology.orchestra.run.smithi104.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-08-15T17:55:00.454 INFO:teuthology.orchestra.run.smithi104.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-08-15T17:55:00.454 INFO:teuthology.orchestra.run.smithi104.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-08-15T17:55:00.454 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-08-15T17:55:00.454 INFO:teuthology.orchestra.run.smithi104.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-08-15T17:55:00.454 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T17:55:00.455 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2024-08-15T17:55:00.455 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-08-15T17:55:00.455 INFO:teuthology.orchestra.run.smithi104.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-08-15T17:55:00.455 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T17:55:00.455 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T17:55:00.455 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T17:55:00.456 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T17:55:00.456 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T17:55:00.456 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2024-08-15T17:55:00.456 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T17:55:00.456 INFO:teuthology.orchestra.run.smithi104.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T17:55:00.457 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T17:55:00.457 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T17:55:00.457 INFO:teuthology.orchestra.run.smithi104.stdout: qttranslations5-l10n socat xmlstarlet 2024-08-15T17:55:00.457 INFO:teuthology.orchestra.run.smithi104.stdout:Suggested packages: 2024-08-15T17:55:00.457 INFO:teuthology.orchestra.run.smithi104.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-08-15T17:55:00.457 INFO:teuthology.orchestra.run.smithi104.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-08-15T17:55:00.458 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-08-15T17:55:00.458 INFO:teuthology.orchestra.run.smithi104.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-08-15T17:55:00.458 INFO:teuthology.orchestra.run.smithi104.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-08-15T17:55:00.458 INFO:teuthology.orchestra.run.smithi104.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-08-15T17:55:00.458 INFO:teuthology.orchestra.run.smithi104.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-08-15T17:55:00.458 INFO:teuthology.orchestra.run.smithi104.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-15T17:55:00.459 INFO:teuthology.orchestra.run.smithi104.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-08-15T17:55:00.459 INFO:teuthology.orchestra.run.smithi104.stdout:Recommended packages: 2024-08-15T17:55:00.459 INFO:teuthology.orchestra.run.smithi104.stdout: btrfs-tools 2024-08-15T17:55:00.483 INFO:teuthology.orchestra.run.smithi038.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T17:55:00.485 INFO:teuthology.orchestra.run.smithi038.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T17:55:00.485 INFO:teuthology.orchestra.run.smithi038.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T17:55:00.485 INFO:teuthology.orchestra.run.smithi038.stdout:The following additional packages will be installed: 2024-08-15T17:55:00.485 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-15T17:55:00.485 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-08-15T17:55:00.485 INFO:teuthology.orchestra.run.smithi038.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-08-15T17:55:00.485 INFO:teuthology.orchestra.run.smithi038.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-08-15T17:55:00.485 INFO:teuthology.orchestra.run.smithi038.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-08-15T17:55:00.488 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-08-15T17:55:00.489 INFO:teuthology.orchestra.run.smithi038.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-08-15T17:55:00.489 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T17:55:00.489 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2024-08-15T17:55:00.489 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-08-15T17:55:00.489 INFO:teuthology.orchestra.run.smithi038.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-08-15T17:55:00.489 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T17:55:00.490 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T17:55:00.490 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T17:55:00.490 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T17:55:00.490 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T17:55:00.491 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2024-08-15T17:55:00.491 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T17:55:00.491 INFO:teuthology.orchestra.run.smithi038.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T17:55:00.491 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T17:55:00.491 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T17:55:00.491 INFO:teuthology.orchestra.run.smithi038.stdout: qttranslations5-l10n socat xmlstarlet 2024-08-15T17:55:00.491 INFO:teuthology.orchestra.run.smithi038.stdout:Suggested packages: 2024-08-15T17:55:00.492 INFO:teuthology.orchestra.run.smithi038.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-08-15T17:55:00.492 INFO:teuthology.orchestra.run.smithi038.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-08-15T17:55:00.492 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-08-15T17:55:00.492 INFO:teuthology.orchestra.run.smithi038.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-08-15T17:55:00.492 INFO:teuthology.orchestra.run.smithi038.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-08-15T17:55:00.492 INFO:teuthology.orchestra.run.smithi038.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-08-15T17:55:00.493 INFO:teuthology.orchestra.run.smithi038.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-08-15T17:55:00.493 INFO:teuthology.orchestra.run.smithi038.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-15T17:55:00.493 INFO:teuthology.orchestra.run.smithi038.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-08-15T17:55:00.493 INFO:teuthology.orchestra.run.smithi038.stdout:Recommended packages: 2024-08-15T17:55:00.493 INFO:teuthology.orchestra.run.smithi038.stdout: btrfs-tools 2024-08-15T17:55:00.575 INFO:teuthology.orchestra.run.smithi104.stdout:The following NEW packages will be installed: 2024-08-15T17:55:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-15T17:55:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-15T17:55:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-08-15T17:55:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-08-15T17:55:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-08-15T17:55:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-08-15T17:55:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-08-15T17:55:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-08-15T17:55:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2024-08-15T17:55:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T17:55:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2024-08-15T17:55:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2024-08-15T17:55:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T17:55:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-08-15T17:55:00.579 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages will be upgraded: 2024-08-15T17:55:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 librbd1 2024-08-15T17:55:00.615 INFO:teuthology.orchestra.run.smithi038.stdout:The following NEW packages will be installed: 2024-08-15T17:55:00.615 INFO:teuthology.orchestra.run.smithi038.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-15T17:55:00.615 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T17:55:00.616 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T17:55:00.617 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T17:55:00.617 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T17:55:00.617 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-08-15T17:55:00.617 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-08-15T17:55:00.617 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T17:55:00.617 INFO:teuthology.orchestra.run.smithi038.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T17:55:00.617 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T17:55:00.617 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T17:55:00.617 INFO:teuthology.orchestra.run.smithi038.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-08-15T17:55:00.617 INFO:teuthology.orchestra.run.smithi038.stdout:The following packages will be upgraded: 2024-08-15T17:55:00.617 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 librbd1 2024-08-15T17:55:00.727 INFO:teuthology.orchestra.run.smithi104.stdout:2 upgraded, 114 newly installed, 0 to remove and 294 not upgraded. 2024-08-15T17:55:00.727 INFO:teuthology.orchestra.run.smithi104.stdout:Need to get 201 MB of archives. 2024-08-15T17:55:00.727 INFO:teuthology.orchestra.run.smithi104.stdout:After this operation, 938 MB of additional disk space will be used. 2024-08-15T17:55:00.728 INFO:teuthology.orchestra.run.smithi104.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1375-g2c991c19-1focal [3,507 kB] 2024-08-15T17:55:00.762 INFO:teuthology.orchestra.run.smithi038.stdout:2 upgraded, 114 newly installed, 0 to remove and 294 not upgraded. 2024-08-15T17:55:00.762 INFO:teuthology.orchestra.run.smithi038.stdout:Need to get 201 MB of archives. 2024-08-15T17:55:00.762 INFO:teuthology.orchestra.run.smithi038.stdout:After this operation, 938 MB of additional disk space will be used. 2024-08-15T17:55:00.762 INFO:teuthology.orchestra.run.smithi038.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1375-g2c991c19-1focal [3,507 kB] 2024-08-15T17:55:00.773 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-08-15T17:55:00.808 INFO:teuthology.orchestra.run.smithi038.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-08-15T17:55:00.856 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-08-15T17:55:00.891 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-08-15T17:55:00.892 INFO:teuthology.orchestra.run.smithi038.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-08-15T17:55:00.903 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1375-g2c991c19-1focal [3,557 kB] 2024-08-15T17:55:00.927 INFO:teuthology.orchestra.run.smithi038.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-08-15T17:55:00.985 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1375-g2c991c19-1focal [761 kB] 2024-08-15T17:55:00.989 INFO:teuthology.orchestra.run.smithi038.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1375-g2c991c19-1focal [3,557 kB] 2024-08-15T17:55:01.016 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-08-15T17:55:01.017 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1375-g2c991c19-1focal [414 kB] 2024-08-15T17:55:01.024 INFO:teuthology.orchestra.run.smithi104.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1375-g2c991c19-1focal [22.6 kB] 2024-08-15T17:55:01.024 INFO:teuthology.orchestra.run.smithi104.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1375-g2c991c19-1focal [200 kB] 2024-08-15T17:55:01.027 INFO:teuthology.orchestra.run.smithi104.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1375-g2c991c19-1focal [49.1 kB] 2024-08-15T17:55:01.028 INFO:teuthology.orchestra.run.smithi104.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1375-g2c991c19-1focal [390 kB] 2024-08-15T17:55:01.051 INFO:teuthology.orchestra.run.smithi038.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-08-15T17:55:01.056 INFO:teuthology.orchestra.run.smithi104.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1375-g2c991c19-1focal [5,612 kB] 2024-08-15T17:55:01.097 INFO:teuthology.orchestra.run.smithi038.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1375-g2c991c19-1focal [761 kB] 2024-08-15T17:55:01.135 INFO:teuthology.orchestra.run.smithi038.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1375-g2c991c19-1focal [414 kB] 2024-08-15T17:55:01.142 INFO:teuthology.orchestra.run.smithi038.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1375-g2c991c19-1focal [22.6 kB] 2024-08-15T17:55:01.142 INFO:teuthology.orchestra.run.smithi038.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1375-g2c991c19-1focal [200 kB] 2024-08-15T17:55:01.144 INFO:teuthology.orchestra.run.smithi038.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1375-g2c991c19-1focal [49.1 kB] 2024-08-15T17:55:01.145 INFO:teuthology.orchestra.run.smithi038.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1375-g2c991c19-1focal [390 kB] 2024-08-15T17:55:01.146 INFO:teuthology.orchestra.run.smithi104.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-08-15T17:55:01.155 INFO:teuthology.orchestra.run.smithi104.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-08-15T17:55:01.177 INFO:teuthology.orchestra.run.smithi038.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1375-g2c991c19-1focal [5,612 kB] 2024-08-15T17:55:01.181 INFO:teuthology.orchestra.run.smithi038.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-08-15T17:55:01.191 INFO:teuthology.orchestra.run.smithi038.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-08-15T17:55:01.198 INFO:teuthology.orchestra.run.smithi104.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-08-15T17:55:01.210 INFO:teuthology.orchestra.run.smithi104.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-08-15T17:55:01.216 INFO:teuthology.orchestra.run.smithi104.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1375-g2c991c19-1focal [116 kB] 2024-08-15T17:55:01.218 INFO:teuthology.orchestra.run.smithi104.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1375-g2c991c19-1focal [479 kB] 2024-08-15T17:55:01.230 INFO:teuthology.orchestra.run.smithi104.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1375-g2c991c19-1focal [26.9 MB] 2024-08-15T17:55:01.235 INFO:teuthology.orchestra.run.smithi038.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-08-15T17:55:01.246 INFO:teuthology.orchestra.run.smithi038.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-08-15T17:55:01.290 INFO:teuthology.orchestra.run.smithi104.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-08-15T17:55:01.295 INFO:teuthology.orchestra.run.smithi104.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-08-15T17:55:01.319 INFO:teuthology.orchestra.run.smithi104.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-08-15T17:55:01.328 INFO:teuthology.orchestra.run.smithi038.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-08-15T17:55:01.332 INFO:teuthology.orchestra.run.smithi038.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-08-15T17:55:01.356 INFO:teuthology.orchestra.run.smithi038.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-08-15T17:55:01.360 INFO:teuthology.orchestra.run.smithi038.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1375-g2c991c19-1focal [116 kB] 2024-08-15T17:55:01.361 INFO:teuthology.orchestra.run.smithi038.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1375-g2c991c19-1focal [479 kB] 2024-08-15T17:55:01.399 INFO:teuthology.orchestra.run.smithi038.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1375-g2c991c19-1focal [26.9 MB] 2024-08-15T17:55:01.447 INFO:teuthology.orchestra.run.smithi104.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-08-15T17:55:01.454 INFO:teuthology.orchestra.run.smithi104.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-08-15T17:55:01.477 INFO:teuthology.orchestra.run.smithi104.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-08-15T17:55:01.482 INFO:teuthology.orchestra.run.smithi038.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-08-15T17:55:01.483 INFO:teuthology.orchestra.run.smithi104.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-08-15T17:55:01.484 INFO:teuthology.orchestra.run.smithi104.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-08-15T17:55:01.492 INFO:teuthology.orchestra.run.smithi104.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-08-15T17:55:01.493 INFO:teuthology.orchestra.run.smithi038.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-08-15T17:55:01.493 INFO:teuthology.orchestra.run.smithi104.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-08-15T17:55:01.494 INFO:teuthology.orchestra.run.smithi104.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-08-15T17:55:01.495 INFO:teuthology.orchestra.run.smithi104.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-08-15T17:55:01.513 INFO:teuthology.orchestra.run.smithi038.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-08-15T17:55:01.521 INFO:teuthology.orchestra.run.smithi038.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-08-15T17:55:01.521 INFO:teuthology.orchestra.run.smithi038.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-08-15T17:55:01.527 INFO:teuthology.orchestra.run.smithi038.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-08-15T17:55:01.528 INFO:teuthology.orchestra.run.smithi038.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-08-15T17:55:01.529 INFO:teuthology.orchestra.run.smithi038.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-08-15T17:55:01.530 INFO:teuthology.orchestra.run.smithi038.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-08-15T17:55:01.556 INFO:teuthology.orchestra.run.smithi104.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-08-15T17:55:01.556 INFO:teuthology.orchestra.run.smithi104.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-08-15T17:55:01.591 INFO:teuthology.orchestra.run.smithi038.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-08-15T17:55:01.592 INFO:teuthology.orchestra.run.smithi038.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-08-15T17:55:01.637 INFO:teuthology.orchestra.run.smithi104.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-08-15T17:55:01.638 INFO:teuthology.orchestra.run.smithi104.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-08-15T17:55:01.639 INFO:teuthology.orchestra.run.smithi104.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-08-15T17:55:01.641 INFO:teuthology.orchestra.run.smithi104.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-15T17:55:01.642 INFO:teuthology.orchestra.run.smithi104.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-08-15T17:55:01.642 INFO:teuthology.orchestra.run.smithi104.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-08-15T17:55:01.644 INFO:teuthology.orchestra.run.smithi104.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-08-15T17:55:01.645 INFO:teuthology.orchestra.run.smithi104.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-08-15T17:55:01.672 INFO:teuthology.orchestra.run.smithi038.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-08-15T17:55:01.673 INFO:teuthology.orchestra.run.smithi038.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-08-15T17:55:01.674 INFO:teuthology.orchestra.run.smithi038.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-08-15T17:55:01.676 INFO:teuthology.orchestra.run.smithi038.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-15T17:55:01.677 INFO:teuthology.orchestra.run.smithi038.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-08-15T17:55:01.678 INFO:teuthology.orchestra.run.smithi038.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-08-15T17:55:01.679 INFO:teuthology.orchestra.run.smithi038.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-08-15T17:55:01.680 INFO:teuthology.orchestra.run.smithi038.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-08-15T17:55:01.718 INFO:teuthology.orchestra.run.smithi104.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-08-15T17:55:01.720 INFO:teuthology.orchestra.run.smithi104.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-08-15T17:55:01.754 INFO:teuthology.orchestra.run.smithi038.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-08-15T17:55:01.756 INFO:teuthology.orchestra.run.smithi038.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-08-15T17:55:01.800 INFO:teuthology.orchestra.run.smithi104.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-08-15T17:55:01.808 INFO:teuthology.orchestra.run.smithi104.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-08-15T17:55:01.809 INFO:teuthology.orchestra.run.smithi104.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-08-15T17:55:01.810 INFO:teuthology.orchestra.run.smithi104.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-08-15T17:55:01.810 INFO:teuthology.orchestra.run.smithi104.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-08-15T17:55:01.813 INFO:teuthology.orchestra.run.smithi104.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-08-15T17:55:01.815 INFO:teuthology.orchestra.run.smithi104.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-08-15T17:55:01.816 INFO:teuthology.orchestra.run.smithi104.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-15T17:55:01.837 INFO:teuthology.orchestra.run.smithi038.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-08-15T17:55:01.845 INFO:teuthology.orchestra.run.smithi038.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-08-15T17:55:01.846 INFO:teuthology.orchestra.run.smithi038.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-08-15T17:55:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-08-15T17:55:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-08-15T17:55:01.850 INFO:teuthology.orchestra.run.smithi038.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-08-15T17:55:01.851 INFO:teuthology.orchestra.run.smithi038.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-08-15T17:55:01.852 INFO:teuthology.orchestra.run.smithi038.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-15T17:55:01.881 INFO:teuthology.orchestra.run.smithi104.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-08-15T17:55:01.884 INFO:teuthology.orchestra.run.smithi104.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-08-15T17:55:01.917 INFO:teuthology.orchestra.run.smithi038.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-08-15T17:55:01.920 INFO:teuthology.orchestra.run.smithi038.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-08-15T17:55:01.963 INFO:teuthology.orchestra.run.smithi104.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-08-15T17:55:01.964 INFO:teuthology.orchestra.run.smithi104.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-08-15T17:55:01.965 INFO:teuthology.orchestra.run.smithi104.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-08-15T17:55:01.966 INFO:teuthology.orchestra.run.smithi104.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-08-15T17:55:01.999 INFO:teuthology.orchestra.run.smithi038.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-08-15T17:55:02.001 INFO:teuthology.orchestra.run.smithi038.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-08-15T17:55:02.002 INFO:teuthology.orchestra.run.smithi038.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-08-15T17:55:02.003 INFO:teuthology.orchestra.run.smithi038.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-08-15T17:55:02.218 INFO:teuthology.orchestra.run.smithi104.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1375-g2c991c19-1focal [5,175 kB] 2024-08-15T17:55:02.245 INFO:teuthology.orchestra.run.smithi104.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-08-15T17:55:02.288 INFO:teuthology.orchestra.run.smithi104.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1375-g2c991c19-1focal [223 kB] 2024-08-15T17:55:02.292 INFO:teuthology.orchestra.run.smithi104.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1375-g2c991c19-1focal [157 kB] 2024-08-15T17:55:02.294 INFO:teuthology.orchestra.run.smithi104.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1375-g2c991c19-1focal [1,594 kB] 2024-08-15T17:55:02.323 INFO:teuthology.orchestra.run.smithi104.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-08-15T17:55:02.326 INFO:teuthology.orchestra.run.smithi104.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-08-15T17:55:02.354 INFO:teuthology.orchestra.run.smithi104.stdout:Get:66 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1375-g2c991c19-1focal [4,687 kB] 2024-08-15T17:55:02.389 INFO:teuthology.orchestra.run.smithi038.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-08-15T17:55:02.391 INFO:teuthology.orchestra.run.smithi104.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-08-15T17:55:02.392 INFO:teuthology.orchestra.run.smithi104.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-08-15T17:55:02.393 INFO:teuthology.orchestra.run.smithi104.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-08-15T17:55:02.394 INFO:teuthology.orchestra.run.smithi104.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-08-15T17:55:02.395 INFO:teuthology.orchestra.run.smithi104.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-08-15T17:55:02.395 INFO:teuthology.orchestra.run.smithi104.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-08-15T17:55:02.395 INFO:teuthology.orchestra.run.smithi104.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-08-15T17:55:02.414 INFO:teuthology.orchestra.run.smithi104.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-08-15T17:55:02.415 INFO:teuthology.orchestra.run.smithi104.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-08-15T17:55:02.457 INFO:teuthology.orchestra.run.smithi104.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1375-g2c991c19-1focal [23.1 MB] 2024-08-15T17:55:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-08-15T17:55:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1375-g2c991c19-1focal [5,175 kB] 2024-08-15T17:55:02.475 INFO:teuthology.orchestra.run.smithi038.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-08-15T17:55:02.492 INFO:teuthology.orchestra.run.smithi038.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-08-15T17:55:02.493 INFO:teuthology.orchestra.run.smithi038.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-08-15T17:55:02.493 INFO:teuthology.orchestra.run.smithi038.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-08-15T17:55:02.493 INFO:teuthology.orchestra.run.smithi038.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-08-15T17:55:02.494 INFO:teuthology.orchestra.run.smithi038.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-08-15T17:55:02.496 INFO:teuthology.orchestra.run.smithi104.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-08-15T17:55:02.501 INFO:teuthology.orchestra.run.smithi104.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-08-15T17:55:02.505 INFO:teuthology.orchestra.run.smithi104.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-08-15T17:55:02.507 INFO:teuthology.orchestra.run.smithi104.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-08-15T17:55:02.509 INFO:teuthology.orchestra.run.smithi104.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-08-15T17:55:02.512 INFO:teuthology.orchestra.run.smithi104.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-08-15T17:55:02.514 INFO:teuthology.orchestra.run.smithi104.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-08-15T17:55:02.514 INFO:teuthology.orchestra.run.smithi104.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-08-15T17:55:02.537 INFO:teuthology.orchestra.run.smithi038.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-08-15T17:55:02.537 INFO:teuthology.orchestra.run.smithi038.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-08-15T17:55:02.544 INFO:teuthology.orchestra.run.smithi038.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1375-g2c991c19-1focal [223 kB] 2024-08-15T17:55:02.548 INFO:teuthology.orchestra.run.smithi038.stdout:Get:71 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1375-g2c991c19-1focal [157 kB] 2024-08-15T17:55:02.555 INFO:teuthology.orchestra.run.smithi038.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1375-g2c991c19-1focal [1,594 kB] 2024-08-15T17:55:02.556 INFO:teuthology.orchestra.run.smithi038.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-08-15T17:55:02.590 INFO:teuthology.orchestra.run.smithi104.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-08-15T17:55:02.594 INFO:teuthology.orchestra.run.smithi104.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-08-15T17:55:02.617 INFO:teuthology.orchestra.run.smithi038.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1375-g2c991c19-1focal [4,687 kB] 2024-08-15T17:55:02.621 INFO:teuthology.orchestra.run.smithi038.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-08-15T17:55:02.622 INFO:teuthology.orchestra.run.smithi038.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-08-15T17:55:02.625 INFO:teuthology.orchestra.run.smithi038.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-08-15T17:55:02.627 INFO:teuthology.orchestra.run.smithi038.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-08-15T17:55:02.629 INFO:teuthology.orchestra.run.smithi038.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-08-15T17:55:02.639 INFO:teuthology.orchestra.run.smithi038.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-08-15T17:55:02.643 INFO:teuthology.orchestra.run.smithi038.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-08-15T17:55:02.646 INFO:teuthology.orchestra.run.smithi038.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-08-15T17:55:02.647 INFO:teuthology.orchestra.run.smithi038.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-08-15T17:55:02.721 INFO:teuthology.orchestra.run.smithi038.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-08-15T17:55:02.726 INFO:teuthology.orchestra.run.smithi038.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-08-15T17:55:02.759 INFO:teuthology.orchestra.run.smithi104.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-08-15T17:55:02.760 INFO:teuthology.orchestra.run.smithi104.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-08-15T17:55:02.760 INFO:teuthology.orchestra.run.smithi104.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-08-15T17:55:02.760 INFO:teuthology.orchestra.run.smithi104.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-08-15T17:55:02.761 INFO:teuthology.orchestra.run.smithi104.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-08-15T17:55:02.761 INFO:teuthology.orchestra.run.smithi104.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-08-15T17:55:02.773 INFO:teuthology.orchestra.run.smithi104.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-15T17:55:02.773 INFO:teuthology.orchestra.run.smithi104.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-08-15T17:55:02.781 INFO:teuthology.orchestra.run.smithi038.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1375-g2c991c19-1focal [23.1 MB] 2024-08-15T17:55:02.803 INFO:teuthology.orchestra.run.smithi104.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-08-15T17:55:02.841 INFO:teuthology.orchestra.run.smithi104.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-08-15T17:55:02.889 INFO:teuthology.orchestra.run.smithi038.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-08-15T17:55:02.891 INFO:teuthology.orchestra.run.smithi038.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-08-15T17:55:02.892 INFO:teuthology.orchestra.run.smithi038.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-08-15T17:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-08-15T17:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-08-15T17:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-08-15T17:55:02.911 INFO:teuthology.orchestra.run.smithi038.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-15T17:55:02.911 INFO:teuthology.orchestra.run.smithi038.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-08-15T17:55:02.912 INFO:teuthology.orchestra.run.smithi038.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-08-15T17:55:02.924 INFO:teuthology.orchestra.run.smithi104.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-08-15T17:55:02.924 INFO:teuthology.orchestra.run.smithi104.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-08-15T17:55:02.928 INFO:teuthology.orchestra.run.smithi104.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-08-15T17:55:02.929 INFO:teuthology.orchestra.run.smithi104.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-08-15T17:55:02.972 INFO:teuthology.orchestra.run.smithi038.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-08-15T17:55:03.055 INFO:teuthology.orchestra.run.smithi038.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-08-15T17:55:03.056 INFO:teuthology.orchestra.run.smithi038.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-08-15T17:55:03.058 INFO:teuthology.orchestra.run.smithi038.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-08-15T17:55:03.059 INFO:teuthology.orchestra.run.smithi038.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-08-15T17:55:03.203 INFO:teuthology.orchestra.run.smithi104.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1375-g2c991c19-1focal [3,944 B] 2024-08-15T17:55:03.204 INFO:teuthology.orchestra.run.smithi104.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1375-g2c991c19-1focal [872 kB] 2024-08-15T17:55:03.216 INFO:teuthology.orchestra.run.smithi104.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1375-g2c991c19-1focal [2,416 kB] 2024-08-15T17:55:03.241 INFO:teuthology.orchestra.run.smithi104.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1375-g2c991c19-1focal [81.4 kB] 2024-08-15T17:55:03.242 INFO:teuthology.orchestra.run.smithi104.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1375-g2c991c19-1focal [120 kB] 2024-08-15T17:55:03.245 INFO:teuthology.orchestra.run.smithi104.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1375-g2c991c19-1focal [6,653 kB] 2024-08-15T17:55:03.453 INFO:teuthology.orchestra.run.smithi104.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1375-g2c991c19-1focal [7,863 kB] 2024-08-15T17:55:03.674 INFO:teuthology.orchestra.run.smithi038.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1375-g2c991c19-1focal [3,944 B] 2024-08-15T17:55:03.674 INFO:teuthology.orchestra.run.smithi038.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1375-g2c991c19-1focal [872 kB] 2024-08-15T17:55:03.685 INFO:teuthology.orchestra.run.smithi104.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1375-g2c991c19-1focal [4,060 B] 2024-08-15T17:55:03.685 INFO:teuthology.orchestra.run.smithi104.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1375-g2c991c19-1focal [53.3 MB] 2024-08-15T17:55:03.687 INFO:teuthology.orchestra.run.smithi038.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1375-g2c991c19-1focal [2,416 kB] 2024-08-15T17:55:03.716 INFO:teuthology.orchestra.run.smithi038.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1375-g2c991c19-1focal [81.4 kB] 2024-08-15T17:55:03.718 INFO:teuthology.orchestra.run.smithi038.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1375-g2c991c19-1focal [120 kB] 2024-08-15T17:55:03.719 INFO:teuthology.orchestra.run.smithi038.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1375-g2c991c19-1focal [6,653 kB] 2024-08-15T17:55:03.997 INFO:teuthology.orchestra.run.smithi038.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1375-g2c991c19-1focal [7,863 kB] 2024-08-15T17:55:04.236 INFO:teuthology.orchestra.run.smithi038.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1375-g2c991c19-1focal [4,060 B] 2024-08-15T17:55:04.237 INFO:teuthology.orchestra.run.smithi038.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1375-g2c991c19-1focal [53.3 MB] 2024-08-15T17:55:05.594 INFO:teuthology.orchestra.run.smithi104.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1375-g2c991c19-1focal [230 kB] 2024-08-15T17:55:05.599 INFO:teuthology.orchestra.run.smithi104.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.7-1375-g2c991c19-1focal [241 kB] 2024-08-15T17:55:05.604 INFO:teuthology.orchestra.run.smithi104.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.7-1375-g2c991c19-1focal [22.0 kB] 2024-08-15T17:55:05.605 INFO:teuthology.orchestra.run.smithi104.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.7-1375-g2c991c19-1focal [13.1 kB] 2024-08-15T17:55:05.605 INFO:teuthology.orchestra.run.smithi104.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1375-g2c991c19-1focal [21.9 kB] 2024-08-15T17:55:05.606 INFO:teuthology.orchestra.run.smithi104.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1375-g2c991c19-1focal [17.1 MB] 2024-08-15T17:55:06.108 INFO:teuthology.orchestra.run.smithi104.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1375-g2c991c19-1focal [80.9 kB] 2024-08-15T17:55:06.116 INFO:teuthology.orchestra.run.smithi038.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1375-g2c991c19-1focal [230 kB] 2024-08-15T17:55:06.121 INFO:teuthology.orchestra.run.smithi038.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.7-1375-g2c991c19-1focal [241 kB] 2024-08-15T17:55:06.126 INFO:teuthology.orchestra.run.smithi038.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.7-1375-g2c991c19-1focal [22.0 kB] 2024-08-15T17:55:06.127 INFO:teuthology.orchestra.run.smithi038.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.7-1375-g2c991c19-1focal [13.1 kB] 2024-08-15T17:55:06.176 INFO:teuthology.orchestra.run.smithi038.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1375-g2c991c19-1focal [21.9 kB] 2024-08-15T17:55:06.176 INFO:teuthology.orchestra.run.smithi038.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1375-g2c991c19-1focal [17.1 MB] 2024-08-15T17:55:06.471 INFO:teuthology.orchestra.run.smithi038.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1375-g2c991c19-1focal [80.9 kB] 2024-08-15T17:55:06.737 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 201 MB in 6s (36.4 MB/s) 2024-08-15T17:55:06.911 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyperclip. 2024-08-15T17:55:06.968 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141718 files and directories currently installed.) 2024-08-15T17:55:06.972 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-08-15T17:55:07.022 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-08-15T17:55:07.133 INFO:teuthology.orchestra.run.smithi038.stdout:Fetched 201 MB in 6s (34.4 MB/s) 2024-08-15T17:55:07.310 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pyperclip. 2024-08-15T17:55:07.342 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-15T17:55:07.361 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-08-15T17:55:07.366 INFO:teuthology.orchestra.run.smithi038.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141718 files and directories currently installed.) 2024-08-15T17:55:07.371 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-08-15T17:55:07.400 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T17:55:07.408 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-08-15T17:55:07.711 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-15T17:55:07.719 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cmd2. 2024-08-15T17:55:07.731 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-08-15T17:55:07.737 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-08-15T17:55:07.769 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T17:55:07.778 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-08-15T17:55:08.055 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-cmd2. 2024-08-15T17:55:08.075 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-08-15T17:55:08.105 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-08-15T17:55:08.147 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-08-15T17:55:08.165 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-08-15T17:55:08.239 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T17:55:08.424 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-08-15T17:55:08.441 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-08-15T17:55:08.525 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T17:55:08.610 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../004-librbd1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:08.760 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librbd1 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T17:55:08.895 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../004-librbd1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:09.020 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking librbd1 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T17:55:09.482 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../005-librados2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:09.606 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librados2 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T17:55:09.690 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../005-librados2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:09.791 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking librados2 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T17:55:10.293 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libcephfs2. 2024-08-15T17:55:10.313 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../006-libcephfs2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:10.351 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:10.487 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libcephfs2. 2024-08-15T17:55:10.504 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../006-libcephfs2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:10.545 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:10.712 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rados. 2024-08-15T17:55:10.728 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../007-python3-rados_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:10.770 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:10.914 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-rados. 2024-08-15T17:55:10.932 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../007-python3-rados_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:10.972 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:11.150 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-15T17:55:11.169 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:11.214 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:11.259 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-15T17:55:11.278 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:11.308 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:11.459 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cephfs. 2024-08-15T17:55:11.475 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../009-python3-cephfs_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:11.534 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:11.561 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-cephfs. 2024-08-15T17:55:11.579 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../009-python3-cephfs_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:11.619 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:11.886 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-15T17:55:11.905 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:11.944 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:11.955 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-15T17:55:11.972 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:12.013 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:12.298 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-prettytable. 2024-08-15T17:55:12.314 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2024-08-15T17:55:12.356 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-08-15T17:55:12.358 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-prettytable. 2024-08-15T17:55:12.374 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2024-08-15T17:55:12.416 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-08-15T17:55:12.642 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rbd. 2024-08-15T17:55:12.658 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../012-python3-rbd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:12.669 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-rbd. 2024-08-15T17:55:12.685 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../012-python3-rbd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:12.700 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:12.727 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:13.053 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-08-15T17:55:13.072 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-08-15T17:55:13.105 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-08-15T17:55:13.111 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T17:55:13.123 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-08-15T17:55:13.163 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T17:55:13.497 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-08-15T17:55:13.515 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2024-08-15T17:55:13.516 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-08-15T17:55:13.533 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2024-08-15T17:55:13.555 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T17:55:13.574 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T17:55:13.934 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-15T17:55:13.936 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-15T17:55:13.950 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-08-15T17:55:13.953 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-08-15T17:55:14.003 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T17:55:14.009 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T17:55:14.395 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-15T17:55:14.397 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-15T17:55:14.413 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-08-15T17:55:14.414 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-08-15T17:55:14.453 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T17:55:14.455 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T17:55:14.773 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-15T17:55:14.790 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-08-15T17:55:14.791 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-15T17:55:14.808 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-08-15T17:55:14.822 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T17:55:14.850 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T17:55:15.293 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-15T17:55:15.312 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:55:15.383 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:55:15.579 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-15T17:55:15.595 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:55:15.647 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:55:16.123 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-15T17:55:16.140 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:55:16.182 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:55:16.238 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-15T17:55:16.253 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:55:16.296 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:55:16.576 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-15T17:55:16.592 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:55:16.634 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:55:16.674 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-15T17:55:16.693 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:55:16.732 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:55:17.004 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-08-15T17:55:17.021 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-08-15T17:55:17.054 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T17:55:17.085 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-08-15T17:55:17.102 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-08-15T17:55:17.168 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T17:55:17.399 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package librgw2. 2024-08-15T17:55:17.415 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../022-librgw2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:17.457 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:17.521 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package librgw2. 2024-08-15T17:55:17.538 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../022-librgw2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:17.579 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:18.218 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-rgw. 2024-08-15T17:55:18.235 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../023-python3-rgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:18.292 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:18.382 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rgw. 2024-08-15T17:55:18.398 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../023-python3-rgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:18.440 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:18.662 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package liboath0. 2024-08-15T17:55:18.678 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-08-15T17:55:18.720 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-08-15T17:55:18.818 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liboath0. 2024-08-15T17:55:18.837 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-08-15T17:55:18.901 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-08-15T17:55:18.981 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libradosstriper1. 2024-08-15T17:55:19.000 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../025-libradosstriper1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:19.031 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:19.154 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libradosstriper1. 2024-08-15T17:55:19.173 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../025-libradosstriper1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:19.204 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:19.518 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-common. 2024-08-15T17:55:19.534 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../026-ceph-common_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:19.576 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:19.690 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-common. 2024-08-15T17:55:19.707 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../026-ceph-common_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:19.748 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:22.181 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-base. 2024-08-15T17:55:22.199 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../027-ceph-base_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:22.250 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:22.354 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-base. 2024-08-15T17:55:22.370 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../027-ceph-base_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:22.422 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:23.127 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-cryptography. 2024-08-15T17:55:23.145 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-08-15T17:55:23.185 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T17:55:23.367 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cryptography. 2024-08-15T17:55:23.383 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-08-15T17:55:23.425 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T17:55:23.581 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-openssl. 2024-08-15T17:55:23.599 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2024-08-15T17:55:23.639 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-08-15T17:55:23.845 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-openssl. 2024-08-15T17:55:23.862 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2024-08-15T17:55:23.928 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-08-15T17:55:23.942 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-packaging. 2024-08-15T17:55:23.958 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../030-python3-packaging_20.3-1_all.deb ... 2024-08-15T17:55:23.991 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-packaging (20.3-1) ... 2024-08-15T17:55:24.219 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-15T17:55:24.235 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../031-ceph-mgr-modules-core_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:24.273 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-packaging. 2024-08-15T17:55:24.293 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../030-python3-packaging_20.3-1_all.deb ... 2024-08-15T17:55:24.294 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:24.342 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-packaging (20.3-1) ... 2024-08-15T17:55:24.575 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-15T17:55:24.593 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../031-ceph-mgr-modules-core_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:24.659 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:24.674 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-15T17:55:24.690 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../032-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-08-15T17:55:24.732 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T17:55:25.060 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-15T17:55:25.063 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-15T17:55:25.076 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../033-python3-cherrypy3_8.9.1-6_all.deb ... 2024-08-15T17:55:25.082 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../032-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-08-15T17:55:25.118 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-08-15T17:55:25.121 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T17:55:25.410 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-15T17:55:25.429 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../033-python3-cherrypy3_8.9.1-6_all.deb ... 2024-08-15T17:55:25.447 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-natsort. 2024-08-15T17:55:25.459 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-08-15T17:55:25.464 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../034-python3-natsort_7.0.1-1_all.deb ... 2024-08-15T17:55:25.505 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-08-15T17:55:25.821 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-natsort. 2024-08-15T17:55:25.841 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../034-python3-natsort_7.0.1-1_all.deb ... 2024-08-15T17:55:25.866 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-jwt. 2024-08-15T17:55:25.879 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-08-15T17:55:25.882 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../035-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-08-15T17:55:25.924 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T17:55:26.219 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-logutils. 2024-08-15T17:55:26.239 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-7_all.deb ... 2024-08-15T17:55:26.240 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-jwt. 2024-08-15T17:55:26.261 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../035-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-08-15T17:55:26.277 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-08-15T17:55:26.299 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T17:55:26.614 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-15T17:55:26.627 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-logutils. 2024-08-15T17:55:26.630 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../037-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-08-15T17:55:26.643 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-7_all.deb ... 2024-08-15T17:55:26.672 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-08-15T17:55:26.676 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-08-15T17:55:26.975 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-mako. 2024-08-15T17:55:26.993 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../038-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-08-15T17:55:27.013 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-15T17:55:27.029 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../037-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-08-15T17:55:27.033 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T17:55:27.104 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-08-15T17:55:27.386 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-15T17:55:27.404 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-15T17:55:27.407 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-mako. 2024-08-15T17:55:27.423 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../038-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-08-15T17:55:27.444 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-15T17:55:27.482 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T17:55:27.722 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-15T17:55:27.738 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-08-15T17:55:27.805 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T17:55:27.818 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-15T17:55:27.837 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-15T17:55:27.877 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-15T17:55:28.083 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-webob. 2024-08-15T17:55:28.100 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.5-2_all.deb ... 2024-08-15T17:55:28.149 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-08-15T17:55:28.204 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-15T17:55:28.224 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-08-15T17:55:28.262 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T17:55:28.502 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-soupsieve. 2024-08-15T17:55:28.522 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../042-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-08-15T17:55:28.560 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T17:55:28.582 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-webob. 2024-08-15T17:55:28.599 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.5-2_all.deb ... 2024-08-15T17:55:28.640 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-08-15T17:55:28.855 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-bs4. 2024-08-15T17:55:28.872 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../043-python3-bs4_4.8.2-1_all.deb ... 2024-08-15T17:55:28.913 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-08-15T17:55:28.993 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-soupsieve. 2024-08-15T17:55:29.009 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../042-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-08-15T17:55:29.051 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T17:55:29.291 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-waitress. 2024-08-15T17:55:29.308 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../044-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-08-15T17:55:29.353 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T17:55:29.362 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-bs4. 2024-08-15T17:55:29.381 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../043-python3-bs4_4.8.2-1_all.deb ... 2024-08-15T17:55:29.420 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-08-15T17:55:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-tempita. 2024-08-15T17:55:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../045-python3-tempita_0.5.2-6_all.deb ... 2024-08-15T17:55:29.736 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-08-15T17:55:29.798 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-waitress. 2024-08-15T17:55:29.815 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../044-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-08-15T17:55:29.859 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T17:55:30.046 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-paste. 2024-08-15T17:55:30.064 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../046-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-08-15T17:55:30.104 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T17:55:30.218 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-tempita. 2024-08-15T17:55:30.237 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../045-python3-tempita_0.5.2-6_all.deb ... 2024-08-15T17:55:30.293 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-08-15T17:55:30.425 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-15T17:55:30.441 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../047-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-08-15T17:55:30.483 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T17:55:30.562 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-paste. 2024-08-15T17:55:30.581 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../046-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-08-15T17:55:30.645 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T17:55:31.002 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-15T17:55:31.019 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../047-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-08-15T17:55:31.060 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T17:55:31.389 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-15T17:55:31.406 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../048-python3-pastedeploy_2.1.0-1_all.deb ... 2024-08-15T17:55:31.528 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-08-15T17:55:31.594 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-15T17:55:31.611 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../048-python3-pastedeploy_2.1.0-1_all.deb ... 2024-08-15T17:55:31.652 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-08-15T17:55:31.955 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-webtest. 2024-08-15T17:55:31.973 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../049-python3-webtest_2.0.32-2_all.deb ... 2024-08-15T17:55:32.022 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-08-15T17:55:32.383 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pecan. 2024-08-15T17:55:32.401 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../050-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-08-15T17:55:32.441 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T17:55:32.608 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-webtest. 2024-08-15T17:55:32.626 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../049-python3-webtest_2.0.32-2_all.deb ... 2024-08-15T17:55:32.666 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-08-15T17:55:32.813 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-15T17:55:32.831 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../051-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-08-15T17:55:32.871 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T17:55:32.986 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pecan. 2024-08-15T17:55:33.005 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../050-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-08-15T17:55:33.044 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T17:55:33.216 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-15T17:55:33.233 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../052-libsqlite3-mod-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:33.274 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:33.407 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-15T17:55:33.423 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../051-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-08-15T17:55:33.465 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T17:55:33.654 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr. 2024-08-15T17:55:33.671 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../053-ceph-mgr_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:33.712 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:33.827 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-15T17:55:33.846 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../052-libsqlite3-mod-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:33.897 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:34.226 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mon. 2024-08-15T17:55:34.245 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../054-ceph-mon_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:34.284 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:34.288 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr. 2024-08-15T17:55:34.306 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../053-ceph-mgr_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:34.346 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:34.882 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mon. 2024-08-15T17:55:34.902 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../054-ceph-mon_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:34.967 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:35.163 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-osd. 2024-08-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../055-ceph-osd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:35.220 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:35.789 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-osd. 2024-08-15T17:55:35.806 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../055-ceph-osd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:35.847 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:37.314 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph. 2024-08-15T17:55:37.332 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../056-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:37.372 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:37.700 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-fuse. 2024-08-15T17:55:37.717 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../057-ceph-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:37.792 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:37.892 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph. 2024-08-15T17:55:37.909 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../056-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:37.950 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:38.220 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mds. 2024-08-15T17:55:38.237 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../058-ceph-mds_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:38.278 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:38.310 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-fuse. 2024-08-15T17:55:38.330 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../057-ceph-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:38.368 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:38.737 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mds. 2024-08-15T17:55:38.757 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../058-ceph-mds_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:38.787 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:38.807 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package cephadm. 2024-08-15T17:55:38.827 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../059-cephadm_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:38.864 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:39.218 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-jinja2. 2024-08-15T17:55:39.234 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../060-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-08-15T17:55:39.276 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T17:55:39.332 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephadm. 2024-08-15T17:55:39.348 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../059-cephadm_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:39.390 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:39.570 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-15T17:55:39.588 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../061-ceph-mgr-cephadm_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:39.628 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:39.718 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-jinja2. 2024-08-15T17:55:39.738 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../060-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-08-15T17:55:39.776 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T17:55:40.023 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-15T17:55:40.041 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2024-08-15T17:55:40.081 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-15T17:55:40.138 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-15T17:55:40.155 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../061-ceph-mgr-cephadm_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:40.196 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:40.351 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libjs-underscore. 2024-08-15T17:55:40.371 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-08-15T17:55:40.409 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T17:55:40.599 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-15T17:55:40.618 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2024-08-15T17:55:40.657 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-15T17:55:40.703 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-08-15T17:55:40.720 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-08-15T17:55:40.761 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T17:55:40.909 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjs-underscore. 2024-08-15T17:55:40.927 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-08-15T17:55:40.993 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T17:55:41.105 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-routes. 2024-08-15T17:55:41.121 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2024-08-15T17:55:41.163 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-08-15T17:55:41.262 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-08-15T17:55:41.277 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-08-15T17:55:41.328 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T17:55:41.525 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-15T17:55:41.541 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:41.583 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:41.614 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-routes. 2024-08-15T17:55:41.633 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2024-08-15T17:55:41.672 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-08-15T17:55:42.017 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-15T17:55:42.034 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:42.075 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:42.997 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-decorator. 2024-08-15T17:55:43.013 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-08-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T17:55:43.399 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-08-15T17:55:43.419 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-08-15T17:55:43.466 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T17:55:43.480 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-decorator. 2024-08-15T17:55:43.499 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-08-15T17:55:43.538 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T17:55:43.768 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-scipy. 2024-08-15T17:55:43.787 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-08-15T17:55:43.827 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-08-15T17:55:43.866 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-08-15T17:55:43.882 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-08-15T17:55:43.924 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T17:55:44.268 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-scipy. 2024-08-15T17:55:44.287 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-08-15T17:55:44.326 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-08-15T17:55:45.145 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-08-15T17:55:45.164 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-08-15T17:55:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T17:55:45.753 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-08-15T17:55:45.765 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-joblib. 2024-08-15T17:55:45.769 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-08-15T17:55:45.781 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2024-08-15T17:55:45.811 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T17:55:45.831 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-08-15T17:55:46.243 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-sklearn. 2024-08-15T17:55:46.263 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-08-15T17:55:46.301 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T17:55:46.397 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-joblib. 2024-08-15T17:55:46.418 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2024-08-15T17:55:46.455 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-08-15T17:55:46.834 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-sklearn. 2024-08-15T17:55:46.851 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-08-15T17:55:46.892 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T17:55:47.335 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-15T17:55:47.353 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:47.385 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:47.866 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-15T17:55:47.883 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:48.008 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:48.529 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-cachetools. 2024-08-15T17:55:48.546 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2024-08-15T17:55:48.587 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-08-15T17:55:48.890 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-15T17:55:48.908 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-08-15T17:55:48.948 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T17:55:49.144 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cachetools. 2024-08-15T17:55:49.160 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2024-08-15T17:55:49.203 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-08-15T17:55:49.310 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-15T17:55:49.327 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-08-15T17:55:49.368 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T17:55:49.530 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-15T17:55:49.550 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-08-15T17:55:49.588 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T17:55:49.679 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-rsa. 2024-08-15T17:55:49.696 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2024-08-15T17:55:49.770 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-rsa (4.0-3) ... 2024-08-15T17:55:49.924 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-15T17:55:49.940 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-08-15T17:55:50.016 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T17:55:50.082 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-google-auth. 2024-08-15T17:55:50.098 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2024-08-15T17:55:50.140 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-08-15T17:55:50.369 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rsa. 2024-08-15T17:55:50.387 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2024-08-15T17:55:50.418 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rsa (4.0-3) ... 2024-08-15T17:55:50.485 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-blinker. 2024-08-15T17:55:50.502 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-08-15T17:55:50.543 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T17:55:50.721 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-google-auth. 2024-08-15T17:55:50.742 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2024-08-15T17:55:50.779 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-08-15T17:55:50.846 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-oauthlib. 2024-08-15T17:55:50.862 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-08-15T17:55:50.904 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T17:55:51.133 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-blinker. 2024-08-15T17:55:51.149 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-08-15T17:55:51.191 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T17:55:51.352 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-15T17:55:51.369 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-08-15T17:55:51.410 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T17:55:51.493 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-oauthlib. 2024-08-15T17:55:51.513 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-08-15T17:55:51.552 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T17:55:51.722 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-websocket. 2024-08-15T17:55:51.738 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-08-15T17:55:51.780 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T17:55:51.990 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-15T17:55:52.010 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-08-15T17:55:52.049 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T17:55:52.125 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-15T17:55:52.144 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-08-15T17:55:52.183 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T17:55:52.394 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-websocket. 2024-08-15T17:55:52.412 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-08-15T17:55:52.477 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T17:55:52.731 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-15T17:55:52.748 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:52.781 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:52.789 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-15T17:55:52.808 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-08-15T17:55:52.856 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T17:55:53.067 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-15T17:55:53.085 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2024-08-15T17:55:53.125 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-08-15T17:55:53.395 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-15T17:55:53.412 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:53.448 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-15T17:55:53.453 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:53.465 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:55:53.498 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:55:53.784 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package jq. 2024-08-15T17:55:53.801 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:55:53.806 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-15T17:55:53.822 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2024-08-15T17:55:53.842 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:55:53.897 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-08-15T17:55:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package socat. 2024-08-15T17:55:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2024-08-15T17:55:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking socat (1.7.3.3-2) ... 2024-08-15T17:55:54.275 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-15T17:55:54.291 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:55:54.333 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:55:54.506 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package xmlstarlet. 2024-08-15T17:55:54.525 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-08-15T17:55:54.564 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-08-15T17:55:54.656 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package jq. 2024-08-15T17:55:54.675 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:55:54.714 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:55:54.936 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-test. 2024-08-15T17:55:54.953 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../090-ceph-test_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:54.992 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package socat. 2024-08-15T17:55:54.993 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:55.008 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2024-08-15T17:55:55.050 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking socat (1.7.3.3-2) ... 2024-08-15T17:55:55.378 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package xmlstarlet. 2024-08-15T17:55:55.398 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-08-15T17:55:55.428 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-08-15T17:55:55.790 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-test. 2024-08-15T17:55:55.810 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../090-ceph-test_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:55:55.848 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:59.499 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-volume. 2024-08-15T17:55:59.518 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../091-ceph-volume_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:55:59.556 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:55:59.998 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package cephfs-mirror. 2024-08-15T17:56:00.015 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../092-cephfs-mirror_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:56:00.056 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:00.460 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-colorama. 2024-08-15T17:56:00.477 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2024-08-15T17:56:00.478 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-volume. 2024-08-15T17:56:00.494 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../091-ceph-volume_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:56:00.518 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-08-15T17:56:00.536 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:00.787 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package cephfs-shell. 2024-08-15T17:56:00.807 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../094-cephfs-shell_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:56:00.845 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:01.004 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephfs-mirror. 2024-08-15T17:56:01.021 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../092-cephfs-mirror_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:56:01.062 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:01.117 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package cephfs-top. 2024-08-15T17:56:01.134 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../095-cephfs-top_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:56:01.175 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:01.428 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package fonts-mathjax. 2024-08-15T17:56:01.431 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-colorama. 2024-08-15T17:56:01.448 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T17:56:01.448 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2024-08-15T17:56:01.486 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:56:01.489 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-08-15T17:56:01.776 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephfs-shell. 2024-08-15T17:56:01.792 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../094-cephfs-shell_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:56:01.842 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:02.039 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-15T17:56:02.057 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../097-libcephfs-dev_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:56:02.097 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:02.145 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephfs-top. 2024-08-15T17:56:02.163 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../095-cephfs-top_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:56:02.194 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:02.465 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package fonts-mathjax. 2024-08-15T17:56:02.482 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T17:56:02.523 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:56:02.534 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package nvme-cli. 2024-08-15T17:56:02.553 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-08-15T17:56:02.610 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T17:56:02.955 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libjs-mathjax. 2024-08-15T17:56:02.973 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../099-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T17:56:03.005 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:56:03.027 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-15T17:56:03.043 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../097-libcephfs-dev_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:56:03.102 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:03.522 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package nvme-cli. 2024-08-15T17:56:03.542 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-08-15T17:56:03.598 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T17:56:03.951 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjs-mathjax. 2024-08-15T17:56:03.967 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../099-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T17:56:04.001 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:56:04.728 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python-natsort-doc. 2024-08-15T17:56:04.750 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../100-python-natsort-doc_7.0.1-1_all.deb ... 2024-08-15T17:56:04.794 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-08-15T17:56:05.139 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-atomicwrites. 2024-08-15T17:56:05.158 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../101-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-08-15T17:56:05.197 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T17:56:05.525 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-attr. 2024-08-15T17:56:05.542 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../102-python3-attr_19.3.0-2_all.deb ... 2024-08-15T17:56:05.583 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-08-15T17:56:05.643 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-natsort-doc. 2024-08-15T17:56:05.664 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../100-python-natsort-doc_7.0.1-1_all.deb ... 2024-08-15T17:56:05.708 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-08-15T17:56:05.922 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-webencodings. 2024-08-15T17:56:05.937 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../103-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-08-15T17:56:05.985 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T17:56:06.061 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-atomicwrites. 2024-08-15T17:56:06.079 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../101-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-08-15T17:56:06.119 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T17:56:06.297 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-html5lib. 2024-08-15T17:56:06.314 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../104-python3-html5lib_1.0.1-2_all.deb ... 2024-08-15T17:56:06.355 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-08-15T17:56:06.447 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-attr. 2024-08-15T17:56:06.464 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../102-python3-attr_19.3.0-2_all.deb ... 2024-08-15T17:56:06.505 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-08-15T17:56:06.716 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-08-15T17:56:06.734 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../105-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-08-15T17:56:06.775 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T17:56:06.882 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-webencodings. 2024-08-15T17:56:06.900 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../103-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-08-15T17:56:06.940 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T17:56:07.236 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pastescript. 2024-08-15T17:56:07.254 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-15T17:56:07.276 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-html5lib. 2024-08-15T17:56:07.293 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../104-python3-html5lib_1.0.1-2_all.deb ... 2024-08-15T17:56:07.294 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-15T17:56:07.334 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-08-15T17:56:07.656 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pluggy. 2024-08-15T17:56:07.674 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2024-08-15T17:56:07.704 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-08-15T17:56:07.714 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-08-15T17:56:07.721 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../105-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-08-15T17:56:07.787 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T17:56:08.026 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-psutil. 2024-08-15T17:56:08.043 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-08-15T17:56:08.109 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T17:56:08.274 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pastescript. 2024-08-15T17:56:08.290 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-15T17:56:08.340 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-15T17:56:08.437 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-py. 2024-08-15T17:56:08.454 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-08-15T17:56:08.495 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T17:56:08.685 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pluggy. 2024-08-15T17:56:08.702 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2024-08-15T17:56:08.743 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-08-15T17:56:08.848 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-15T17:56:08.866 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-08-15T17:56:08.906 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T17:56:09.063 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-psutil. 2024-08-15T17:56:09.083 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-08-15T17:56:09.121 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T17:56:09.209 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pytest. 2024-08-15T17:56:09.227 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2024-08-15T17:56:09.267 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-08-15T17:56:09.457 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-py. 2024-08-15T17:56:09.476 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-08-15T17:56:09.548 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T17:56:09.638 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-simplejson. 2024-08-15T17:56:09.657 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-08-15T17:56:09.695 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T17:56:09.835 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-15T17:56:09.855 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-08-15T17:56:09.901 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T17:56:09.948 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-15T17:56:09.969 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-08-15T17:56:10.007 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T17:56:10.238 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pytest. 2024-08-15T17:56:10.256 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2024-08-15T17:56:10.295 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-08-15T17:56:10.674 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-simplejson. 2024-08-15T17:56:10.690 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-08-15T17:56:10.722 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package radosgw. 2024-08-15T17:56:10.732 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T17:56:10.744 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../114-radosgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:56:10.780 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:11.034 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-15T17:56:11.052 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-08-15T17:56:11.092 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T17:56:11.899 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package radosgw. 2024-08-15T17:56:11.917 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../114-radosgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:56:11.957 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:12.550 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package rbd-fuse. 2024-08-15T17:56:12.570 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../115-rbd-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:56:12.608 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:12.909 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-attr (19.3.0-2) ... 2024-08-15T17:56:13.213 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T17:56:13.339 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T17:56:13.465 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T17:56:13.667 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:13.736 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package rbd-fuse. 2024-08-15T17:56:13.752 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../115-rbd-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:56:13.818 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:14.165 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-attr (19.3.0-2) ... 2024-08-15T17:56:14.223 INFO:teuthology.orchestra.run.smithi038.stdout:Adding system user cephadm....done 2024-08-15T17:56:14.409 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:56:14.448 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T17:56:14.537 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T17:56:14.599 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T17:56:14.725 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T17:56:14.879 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-15T17:56:14.952 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:15.147 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:56:15.264 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-08-15T17:56:15.390 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T17:56:15.554 INFO:teuthology.orchestra.run.smithi104.stdout:Adding system user cephadm....done 2024-08-15T17:56:15.701 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-08-15T17:56:15.726 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:56:15.852 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T17:56:16.120 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T17:56:16.245 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-15T17:56:16.380 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-08-15T17:56:16.522 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:56:16.648 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-08-15T17:56:16.673 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-08-15T17:56:16.815 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T17:56:16.941 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T17:56:17.151 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-08-15T17:56:17.250 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T17:56:17.535 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:17.545 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T17:56:17.836 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T17:56:17.871 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-08-15T17:56:18.172 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-08-15T17:56:18.457 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T17:56:18.545 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-08-15T17:56:18.846 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-08-15T17:56:19.124 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T17:56:19.156 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T17:56:19.417 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:19.432 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T17:56:19.558 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T17:56:19.735 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T17:56:19.885 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-08-15T17:56:19.942 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-08-15T17:56:20.187 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-08-15T17:56:20.277 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T17:56:20.471 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T17:56:20.561 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T17:56:20.772 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T17:56:20.912 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T17:56:20.923 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T17:56:21.214 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T17:56:21.299 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-08-15T17:56:21.557 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-08-15T17:56:21.625 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T17:56:21.833 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-packaging (20.3-1) ... 2024-08-15T17:56:21.943 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T17:56:22.147 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T17:56:22.276 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T17:56:22.302 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T17:56:22.402 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T17:56:22.578 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T17:56:22.712 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T17:56:22.937 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-08-15T17:56:23.122 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-08-15T17:56:23.238 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-packaging (20.3-1) ... 2024-08-15T17:56:23.423 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T17:56:23.556 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T17:56:23.707 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T17:56:23.808 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T17:56:23.833 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T17:56:24.126 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T17:56:24.185 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up socat (1.7.3.3-2) ... 2024-08-15T17:56:24.328 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:24.519 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-08-15T17:56:24.654 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:56:24.788 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-08-15T17:56:24.812 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T17:56:25.140 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T17:56:25.171 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T17:56:25.265 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-08-15T17:56:25.391 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-08-15T17:56:25.531 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up socat (1.7.3.3-2) ... 2024-08-15T17:56:25.649 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:25.701 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T17:56:25.975 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:56:26.027 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:56:26.109 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-08-15T17:56:26.153 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T17:56:26.279 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T17:56:26.527 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T17:56:26.648 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T17:56:26.653 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-08-15T17:56:26.779 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-08-15T17:56:26.949 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-08-15T17:56:27.105 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T17:56:27.258 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T17:56:27.376 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-15T17:56:27.415 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:56:27.541 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T17:56:27.660 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-08-15T17:56:27.694 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T17:56:27.953 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T17:56:28.010 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T17:56:28.271 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T17:56:28.319 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-08-15T17:56:28.654 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T17:56:28.717 INFO:teuthology.orchestra.run.smithi038.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-08-15T17:56:28.779 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-15T17:56:28.990 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-08-15T17:56:29.064 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-08-15T17:56:29.116 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T17:56:29.365 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T17:56:29.467 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-08-15T17:56:29.649 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T17:56:29.827 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T17:56:29.978 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T17:56:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-08-15T17:56:30.104 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-08-15T17:56:30.326 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-08-15T17:56:30.447 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T17:56:30.478 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T17:56:30.740 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up librados2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:30.829 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-08-15T17:56:30.866 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-08-15T17:56:31.197 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T17:56:31.323 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T17:56:31.465 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-08-15T17:56:31.833 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T17:56:32.109 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librados2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:32.260 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-08-15T17:56:32.388 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:56:32.640 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-08-15T17:56:33.075 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-08-15T17:56:33.376 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-08-15T17:56:33.694 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:56:33.748 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:33.828 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-08-15T17:56:33.874 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:56:34.017 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-08-15T17:56:34.171 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:34.322 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T17:56:34.393 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-08-15T17:56:34.658 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:34.688 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-08-15T17:56:34.784 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T17:56:35.006 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:56:35.132 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-08-15T17:56:35.143 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T17:56:35.458 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:35.575 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T17:56:35.919 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:36.078 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T17:56:36.249 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up librbd1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:36.375 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T17:56:36.413 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T17:56:36.501 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-08-15T17:56:36.619 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:56:36.744 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-rsa (4.0-3) ... 2024-08-15T17:56:37.037 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T17:56:37.163 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-08-15T17:56:37.460 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librbd1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:37.550 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:37.586 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T17:56:37.701 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:37.712 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-08-15T17:56:37.871 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:56:37.926 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T17:56:37.926 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T17:56:37.997 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rsa (4.0-3) ... 2024-08-15T17:56:38.273 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T17:56:38.369 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:38.399 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-08-15T17:56:38.487 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T17:56:38.802 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:38.830 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-15T17:56:38.953 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:39.140 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T17:56:39.186 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T17:56:39.186 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T17:56:39.492 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:39.593 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:39.655 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:39.694 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:39.781 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T17:56:39.829 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:39.955 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:40.090 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-routes (2.4.1-2) ... 2024-08-15T17:56:40.108 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-15T17:56:40.374 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-08-15T17:56:40.417 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T17:56:40.692 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:40.761 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:40.810 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:40.888 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:40.936 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:41.047 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:41.062 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T17:56:41.173 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:41.324 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:41.450 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-routes (2.4.1-2) ... 2024-08-15T17:56:41.751 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-08-15T17:56:42.007 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:42.061 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:42.187 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:42.313 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:42.472 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T17:56:42.576 INFO:teuthology.orchestra.run.smithi038.stdout:Adding group ceph....done 2024-08-15T17:56:42.953 INFO:teuthology.orchestra.run.smithi038.stdout:Adding system user ceph....done 2024-08-15T17:56:43.130 INFO:teuthology.orchestra.run.smithi038.stdout:Setting system user ceph properties....done 2024-08-15T17:56:43.152 INFO:teuthology.orchestra.run.smithi038.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-15T17:56:43.302 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-15T17:56:43.425 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:43.622 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-15T17:56:43.987 INFO:teuthology.orchestra.run.smithi104.stdout:Adding group ceph....done 2024-08-15T17:56:44.043 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:44.273 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T17:56:44.273 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T17:56:44.438 INFO:teuthology.orchestra.run.smithi104.stdout:Adding system user ceph....done 2024-08-15T17:56:44.631 INFO:teuthology.orchestra.run.smithi104.stdout:Setting system user ceph properties....done 2024-08-15T17:56:44.650 INFO:teuthology.orchestra.run.smithi104.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-15T17:56:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:44.796 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-15T17:56:44.853 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:45.104 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-15T17:56:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T17:56:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T17:56:45.577 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:45.787 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T17:56:45.787 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T17:56:45.789 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:46.152 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-15T17:56:46.261 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:46.387 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:46.624 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:46.658 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T17:56:46.659 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T17:56:46.853 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T17:56:46.854 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T17:56:47.301 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:47.322 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:47.537 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T17:56:47.537 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T17:56:47.687 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-15T17:56:47.969 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:48.149 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:48.378 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T17:56:48.378 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T17:56:48.452 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T17:56:48.452 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T17:56:48.842 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:48.921 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:49.049 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:49.075 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T17:56:49.075 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T17:56:49.527 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:49.601 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:49.901 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T17:56:49.901 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T17:56:50.378 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:50.513 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T17:56:50.513 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T17:56:50.521 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:50.647 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:50.789 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:50.916 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T17:56:50.982 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:51.101 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T17:56:51.105 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:51.273 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:51.369 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T17:56:51.500 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T17:56:51.500 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T17:56:51.974 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:52.117 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:52.234 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:52.402 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:56:52.537 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T17:56:52.722 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T17:56:53.040 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T17:56:54.314 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-08-15T17:56:54.977 INFO:teuthology.orchestra.run.smithi038.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T17:56:54.980 DEBUG:teuthology.orchestra.run.smithi038:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-08-15T17:56:55.058 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2024-08-15T17:56:55.281 INFO:teuthology.orchestra.run.smithi038.stdout:Building dependency tree... 2024-08-15T17:56:55.282 INFO:teuthology.orchestra.run.smithi038.stdout:Reading state information... 2024-08-15T17:56:55.516 INFO:teuthology.orchestra.run.smithi038.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-08-15T17:56:55.516 INFO:teuthology.orchestra.run.smithi038.stdout:flex is already the newest version (2.6.4-6.2). 2024-08-15T17:56:55.516 INFO:teuthology.orchestra.run.smithi038.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-08-15T17:56:55.516 INFO:teuthology.orchestra.run.smithi038.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-08-15T17:56:55.516 INFO:teuthology.orchestra.run.smithi038.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-08-15T17:56:55.517 INFO:teuthology.orchestra.run.smithi038.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T17:56:55.517 INFO:teuthology.orchestra.run.smithi038.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T17:56:55.517 INFO:teuthology.orchestra.run.smithi038.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T17:56:55.517 INFO:teuthology.orchestra.run.smithi038.stdout:The following additional packages will be installed: 2024-08-15T17:56:55.518 INFO:teuthology.orchestra.run.smithi038.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-08-15T17:56:55.518 INFO:teuthology.orchestra.run.smithi038.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2024-08-15T17:56:55.518 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2024-08-15T17:56:55.518 INFO:teuthology.orchestra.run.smithi038.stdout: usb-modeswitch-data wpasupplicant 2024-08-15T17:56:55.520 INFO:teuthology.orchestra.run.smithi038.stdout:Suggested packages: 2024-08-15T17:56:55.520 INFO:teuthology.orchestra.run.smithi038.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2024-08-15T17:56:55.520 INFO:teuthology.orchestra.run.smithi038.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-08-15T17:56:55.576 INFO:teuthology.orchestra.run.smithi038.stdout:The following NEW packages will be installed: 2024-08-15T17:56:55.577 INFO:teuthology.orchestra.run.smithi038.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-08-15T17:56:55.577 INFO:teuthology.orchestra.run.smithi038.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2024-08-15T17:56:55.577 INFO:teuthology.orchestra.run.smithi038.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-08-15T17:56:55.578 INFO:teuthology.orchestra.run.smithi038.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2024-08-15T17:56:55.578 INFO:teuthology.orchestra.run.smithi038.stdout:The following packages will be upgraded: 2024-08-15T17:56:55.580 INFO:teuthology.orchestra.run.smithi038.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-08-15T17:56:55.636 INFO:teuthology.orchestra.run.smithi038.stdout:6 upgraded, 23 newly installed, 0 to remove and 288 not upgraded. 2024-08-15T17:56:55.636 INFO:teuthology.orchestra.run.smithi038.stdout:Need to get 10.4 MB of archives. 2024-08-15T17:56:55.636 INFO:teuthology.orchestra.run.smithi038.stdout:After this operation, 33.5 MB of additional disk space will be used. 2024-08-15T17:56:55.636 INFO:teuthology.orchestra.run.smithi038.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-08-15T17:56:55.767 INFO:teuthology.orchestra.run.smithi038.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-08-15T17:56:55.768 INFO:teuthology.orchestra.run.smithi038.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-08-15T17:56:55.769 INFO:teuthology.orchestra.run.smithi038.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-08-15T17:56:55.778 INFO:teuthology.orchestra.run.smithi038.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-08-15T17:56:55.778 INFO:teuthology.orchestra.run.smithi038.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-08-15T17:56:55.791 INFO:teuthology.orchestra.run.smithi038.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-08-15T17:56:55.792 INFO:teuthology.orchestra.run.smithi038.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-08-15T17:56:55.793 INFO:teuthology.orchestra.run.smithi038.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-08-15T17:56:55.794 INFO:teuthology.orchestra.run.smithi038.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-08-15T17:56:55.795 INFO:teuthology.orchestra.run.smithi038.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-08-15T17:56:55.800 INFO:teuthology.orchestra.run.smithi038.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-08-15T17:56:55.801 INFO:teuthology.orchestra.run.smithi038.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-08-15T17:56:55.803 INFO:teuthology.orchestra.run.smithi038.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-08-15T17:56:55.803 INFO:teuthology.orchestra.run.smithi038.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-08-15T17:56:55.817 INFO:teuthology.orchestra.run.smithi038.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-08-15T17:56:55.818 INFO:teuthology.orchestra.run.smithi038.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-08-15T17:56:55.824 INFO:teuthology.orchestra.run.smithi038.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-08-15T17:56:55.832 INFO:teuthology.orchestra.run.smithi038.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-08-15T17:56:55.832 INFO:teuthology.orchestra.run.smithi038.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-08-15T17:56:55.847 INFO:teuthology.orchestra.run.smithi038.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-08-15T17:56:55.847 INFO:teuthology.orchestra.run.smithi038.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-08-15T17:56:55.854 INFO:teuthology.orchestra.run.smithi038.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-08-15T17:56:55.865 INFO:teuthology.orchestra.run.smithi038.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-08-15T17:56:55.880 INFO:teuthology.orchestra.run.smithi038.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-08-15T17:56:55.884 INFO:teuthology.orchestra.run.smithi038.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-08-15T17:56:55.884 INFO:teuthology.orchestra.run.smithi038.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-08-15T17:56:55.884 INFO:teuthology.orchestra.run.smithi038.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-08-15T17:56:55.885 INFO:teuthology.orchestra.run.smithi038.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-08-15T17:56:55.918 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-08-15T17:56:56.223 INFO:teuthology.orchestra.run.smithi038.stdout:Preconfiguring packages ... 2024-08-15T17:56:56.354 INFO:teuthology.orchestra.run.smithi038.stdout:Fetched 10.4 MB in 0s (35.3 MB/s) 2024-08-15T17:56:56.524 INFO:teuthology.orchestra.run.smithi104.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T17:56:56.526 DEBUG:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-08-15T17:56:56.597 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2024-08-15T17:56:56.632 INFO:teuthology.orchestra.run.smithi038.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:56:56.637 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:56:56.813 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2024-08-15T17:56:56.813 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2024-08-15T17:56:56.850 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:56:57.034 INFO:teuthology.orchestra.run.smithi104.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-08-15T17:56:57.034 INFO:teuthology.orchestra.run.smithi104.stdout:flex is already the newest version (2.6.4-6.2). 2024-08-15T17:56:57.034 INFO:teuthology.orchestra.run.smithi104.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-08-15T17:56:57.034 INFO:teuthology.orchestra.run.smithi104.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-08-15T17:56:57.034 INFO:teuthology.orchestra.run.smithi104.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-08-15T17:56:57.034 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T17:56:57.035 INFO:teuthology.orchestra.run.smithi104.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T17:56:57.035 INFO:teuthology.orchestra.run.smithi104.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T17:56:57.035 INFO:teuthology.orchestra.run.smithi104.stdout:The following additional packages will be installed: 2024-08-15T17:56:57.035 INFO:teuthology.orchestra.run.smithi104.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-08-15T17:56:57.035 INFO:teuthology.orchestra.run.smithi104.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2024-08-15T17:56:57.036 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2024-08-15T17:56:57.036 INFO:teuthology.orchestra.run.smithi104.stdout: usb-modeswitch-data wpasupplicant 2024-08-15T17:56:57.037 INFO:teuthology.orchestra.run.smithi104.stdout:Suggested packages: 2024-08-15T17:56:57.037 INFO:teuthology.orchestra.run.smithi104.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2024-08-15T17:56:57.038 INFO:teuthology.orchestra.run.smithi104.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-08-15T17:56:57.089 INFO:teuthology.orchestra.run.smithi104.stdout:The following NEW packages will be installed: 2024-08-15T17:56:57.090 INFO:teuthology.orchestra.run.smithi104.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-08-15T17:56:57.090 INFO:teuthology.orchestra.run.smithi104.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2024-08-15T17:56:57.090 INFO:teuthology.orchestra.run.smithi104.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-08-15T17:56:57.091 INFO:teuthology.orchestra.run.smithi104.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2024-08-15T17:56:57.091 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages will be upgraded: 2024-08-15T17:56:57.093 INFO:teuthology.orchestra.run.smithi104.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-08-15T17:56:57.275 INFO:teuthology.orchestra.run.smithi104.stdout:6 upgraded, 23 newly installed, 0 to remove and 288 not upgraded. 2024-08-15T17:56:57.276 INFO:teuthology.orchestra.run.smithi104.stdout:Need to get 10.4 MB of archives. 2024-08-15T17:56:57.276 INFO:teuthology.orchestra.run.smithi104.stdout:After this operation, 33.5 MB of additional disk space will be used. 2024-08-15T17:56:57.276 INFO:teuthology.orchestra.run.smithi104.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-08-15T17:56:57.888 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-08-15T17:56:57.889 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-08-15T17:56:57.890 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-08-15T17:56:57.903 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-08-15T17:56:57.905 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-08-15T17:56:58.002 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-08-15T17:56:58.005 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-08-15T17:56:58.009 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-08-15T17:56:58.012 INFO:teuthology.orchestra.run.smithi104.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-08-15T17:56:58.016 INFO:teuthology.orchestra.run.smithi104.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-08-15T17:56:58.017 INFO:teuthology.orchestra.run.smithi104.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-08-15T17:56:58.050 INFO:teuthology.orchestra.run.smithi104.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-08-15T17:56:58.052 INFO:teuthology.orchestra.run.smithi104.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-08-15T17:56:58.055 INFO:teuthology.orchestra.run.smithi104.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-08-15T17:56:58.056 INFO:teuthology.orchestra.run.smithi104.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-08-15T17:56:58.131 INFO:teuthology.orchestra.run.smithi104.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-08-15T17:56:58.132 INFO:teuthology.orchestra.run.smithi104.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-08-15T17:56:58.141 INFO:teuthology.orchestra.run.smithi104.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-08-15T17:56:58.157 INFO:teuthology.orchestra.run.smithi104.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-08-15T17:56:58.157 INFO:teuthology.orchestra.run.smithi104.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-08-15T17:56:58.196 INFO:teuthology.orchestra.run.smithi104.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-08-15T17:56:58.212 INFO:teuthology.orchestra.run.smithi104.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-08-15T17:56:58.234 INFO:teuthology.orchestra.run.smithi104.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-08-15T17:56:58.262 INFO:teuthology.orchestra.run.smithi104.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-08-15T17:56:58.321 INFO:teuthology.orchestra.run.smithi104.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-08-15T17:56:58.330 INFO:teuthology.orchestra.run.smithi104.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-08-15T17:56:58.330 INFO:teuthology.orchestra.run.smithi104.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-08-15T17:56:58.331 INFO:teuthology.orchestra.run.smithi104.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-08-15T17:56:58.331 INFO:teuthology.orchestra.run.smithi104.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-08-15T17:56:58.652 INFO:teuthology.orchestra.run.smithi104.stdout:Preconfiguring packages ... 2024-08-15T17:56:58.801 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 10.4 MB in 1s (8,477 kB/s) 2024-08-15T17:56:59.104 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:56:59.108 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:56:59.261 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:56:59.405 INFO:teuthology.orchestra.run.smithi038.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:56:59.410 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:56:59.565 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:57:00.048 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:57:00.281 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:57:00.601 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-08-15T17:57:00.606 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T17:57:01.053 INFO:teuthology.orchestra.run.smithi038.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:57:01.058 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:57:01.189 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T17:57:01.665 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:57:01.769 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T17:57:02.031 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:57:02.035 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:57:02.184 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:57:02.250 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T17:57:02.424 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-08-15T17:57:02.590 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:57:02.765 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:57:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-08-15T17:57:02.913 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-08-15T17:57:02.945 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-08-15T17:57:03.126 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T17:57:03.315 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package dump. 2024-08-15T17:57:03.332 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-08-15T17:57:03.373 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking dump (0.4b46-6) ... 2024-08-15T17:57:03.634 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:57:03.638 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:57:03.668 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package indent. 2024-08-15T17:57:03.686 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:57:03.752 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T17:57:03.806 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T17:57:04.055 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-08-15T17:57:04.072 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-08-15T17:57:04.104 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T17:57:04.311 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:57:04.445 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T17:57:04.450 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-08-15T17:57:04.469 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-08-15T17:57:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T17:57:04.769 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-08-15T17:57:04.786 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:57:04.828 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:57:04.950 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T17:57:05.107 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-08-15T17:57:05.324 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-08-15T17:57:05.342 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:05.374 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:57:05.618 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libmbim-proxy. 2024-08-15T17:57:05.637 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:05.668 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:57:06.013 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-08-15T17:57:06.030 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:06.071 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:57:06.430 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-08-15T17:57:06.433 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libndp0:amd64. 2024-08-15T17:57:06.450 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-08-15T17:57:06.450 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-08-15T17:57:06.491 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T17:57:06.658 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-08-15T17:57:06.836 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libnm0:amd64. 2024-08-15T17:57:06.856 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T17:57:06.894 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T17:57:07.083 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package dump. 2024-08-15T17:57:07.100 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-08-15T17:57:07.175 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking dump (0.4b46-6) ... 2024-08-15T17:57:07.305 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-08-15T17:57:07.322 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:07.364 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:57:07.478 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package indent. 2024-08-15T17:57:07.495 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:57:07.553 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T17:57:07.700 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libqmi-proxy. 2024-08-15T17:57:07.717 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:07.766 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:57:07.898 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-08-15T17:57:07.915 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-08-15T17:57:07.956 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T17:57:08.061 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-08-15T17:57:08.079 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T17:57:08.119 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:57:08.317 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-08-15T17:57:08.335 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-08-15T17:57:08.376 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T17:57:08.664 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-08-15T17:57:08.670 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-08-15T17:57:08.682 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2024-08-15T17:57:08.689 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:57:08.722 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-08-15T17:57:08.728 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:57:09.101 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package modemmanager. 2024-08-15T17:57:09.106 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-08-15T17:57:09.118 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:09.123 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:09.156 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:57:09.170 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:57:09.476 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libmbim-proxy. 2024-08-15T17:57:09.492 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:09.534 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:57:09.832 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package wpasupplicant. 2024-08-15T17:57:09.850 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-08-15T17:57:09.890 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T17:57:09.895 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-08-15T17:57:09.916 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:09.978 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:57:10.373 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libndp0:amd64. 2024-08-15T17:57:10.391 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-08-15T17:57:10.434 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T17:57:10.661 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package network-manager. 2024-08-15T17:57:10.679 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T17:57:10.776 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libnm0:amd64. 2024-08-15T17:57:10.792 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T17:57:10.815 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T17:57:10.834 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T17:57:11.254 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-08-15T17:57:11.271 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:11.312 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:57:11.556 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ppp. 2024-08-15T17:57:11.574 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-08-15T17:57:11.656 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqmi-proxy. 2024-08-15T17:57:11.673 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:11.739 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:57:11.835 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T17:57:12.161 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package pptp-linux. 2024-08-15T17:57:12.181 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-08-15T17:57:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-08-15T17:57:12.606 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package network-manager-pptp. 2024-08-15T17:57:12.625 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-08-15T17:57:12.656 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-08-15T17:57:12.677 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T17:57:12.682 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-08-15T17:57:12.810 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:57:12.943 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-08-15T17:57:12.964 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2024-08-15T17:57:13.001 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-08-15T17:57:13.354 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package usb-modeswitch. 2024-08-15T17:57:13.373 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-08-15T17:57:13.423 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T17:57:13.548 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-08-15T17:57:13.567 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2024-08-15T17:57:13.631 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-08-15T17:57:13.757 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:57:14.035 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package modemmanager. 2024-08-15T17:57:14.051 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:57:14.060 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-08-15T17:57:14.112 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:57:14.186 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T17:57:14.312 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T17:57:14.737 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T17:57:14.737 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T17:57:14.757 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package wpasupplicant. 2024-08-15T17:57:14.775 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-08-15T17:57:14.815 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T17:57:15.207 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T17:57:15.333 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-08-15T17:57:15.412 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package network-manager. 2024-08-15T17:57:15.431 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T17:57:15.459 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:57:15.493 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T17:57:15.585 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-08-15T17:57:15.814 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T17:57:16.106 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T17:57:16.270 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ppp. 2024-08-15T17:57:16.289 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-08-15T17:57:16.548 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T17:57:16.899 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package pptp-linux. 2024-08-15T17:57:16.916 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-08-15T17:57:16.958 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-08-15T17:57:17.047 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-08-15T17:57:17.286 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up dump (0.4b46-6) ... 2024-08-15T17:57:17.353 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package network-manager-pptp. 2024-08-15T17:57:17.370 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-08-15T17:57:17.382 INFO:teuthology.orchestra.run.smithi038.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-08-15T17:57:17.429 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-08-15T17:57:17.447 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:57:17.573 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T17:57:17.699 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T17:57:17.705 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-08-15T17:57:17.723 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2024-08-15T17:57:17.763 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-08-15T17:57:17.825 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T17:57:18.035 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T17:57:18.153 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:57:18.254 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:57:18.355 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-08-15T17:57:18.514 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:57:18.674 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T17:57:18.912 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package usb-modeswitch. 2024-08-15T17:57:18.930 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-08-15T17:57:19.012 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T17:57:19.306 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:57:19.375 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:19.375 INFO:teuthology.orchestra.run.smithi038.stdout:The following network interfaces were found in /etc/network/interfaces 2024-08-15T17:57:19.375 INFO:teuthology.orchestra.run.smithi038.stdout:which means they are currently configured by ifupdown: 2024-08-15T17:57:19.375 INFO:teuthology.orchestra.run.smithi038.stdout:- ens1f0 2024-08-15T17:57:19.375 INFO:teuthology.orchestra.run.smithi038.stdout:If you want to manage those interfaces with NetworkManager instead 2024-08-15T17:57:19.376 INFO:teuthology.orchestra.run.smithi038.stdout:remove their configuration from /etc/network/interfaces. 2024-08-15T17:57:19.376 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:19.539 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-08-15T17:57:19.649 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-08-15T17:57:19.800 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T17:57:19.860 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-08-15T17:57:19.926 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T17:57:20.355 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T17:57:20.355 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T17:57:20.391 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-08-15T17:57:20.829 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T17:57:20.886 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:57:20.955 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-08-15T17:57:20.995 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-08-15T17:57:21.102 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:57:21.232 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-08-15T17:57:21.358 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T17:57:21.517 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T17:57:21.547 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:57:21.723 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:57:21.849 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:57:21.975 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:57:22.245 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T17:57:22.246 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T17:57:22.532 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-08-15T17:57:22.745 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-08-15T17:57:22.773 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up dump (0.4b46-6) ... 2024-08-15T17:57:22.854 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-08-15T17:57:22.893 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-08-15T17:57:22.958 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:57:22.982 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T17:57:23.084 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T17:57:23.131 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T17:57:23.235 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T17:57:23.353 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T17:57:23.441 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T17:57:23.521 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T17:57:23.680 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:57:23.806 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:57:23.957 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-08-15T17:57:24.276 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:57:24.768 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T17:57:25.452 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:25.453 INFO:teuthology.orchestra.run.smithi104.stdout:The following network interfaces were found in /etc/network/interfaces 2024-08-15T17:57:25.453 INFO:teuthology.orchestra.run.smithi104.stdout:which means they are currently configured by ifupdown: 2024-08-15T17:57:25.453 INFO:teuthology.orchestra.run.smithi104.stdout:- enp3s0f1 2024-08-15T17:57:25.453 INFO:teuthology.orchestra.run.smithi104.stdout:If you want to manage those interfaces with NetworkManager instead 2024-08-15T17:57:25.453 INFO:teuthology.orchestra.run.smithi104.stdout:remove their configuration from /etc/network/interfaces. 2024-08-15T17:57:25.453 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:25.610 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-08-15T17:57:25.936 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-08-15T17:57:26.461 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-08-15T17:57:26.971 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:57:27.114 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-08-15T17:57:27.290 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:57:27.361 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-08-15T17:57:27.416 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:57:27.461 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-08-15T17:57:27.570 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:57:27.687 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:57:27.958 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T17:57:27.958 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T17:57:28.353 INFO:teuthology.orchestra.run.smithi038.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T17:57:28.358 DEBUG:teuthology.parallel:result is None 2024-08-15T17:57:28.490 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-08-15T17:57:28.607 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-08-15T17:57:28.733 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T17:57:28.884 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T17:57:29.203 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T17:57:32.930 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-08-15T17:57:33.038 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-08-15T17:57:33.898 INFO:teuthology.orchestra.run.smithi104.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T17:57:33.902 DEBUG:teuthology.parallel:result is None 2024-08-15T17:57:33.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T17:57:34.056 DEBUG:teuthology.orchestra.run.smithi038:> dpkg-query -W -f '${Version}' ceph 2024-08-15T17:57:34.079 INFO:teuthology.orchestra.run.smithi038.stdout:17.2.7-1375-g2c991c19-1focal 2024-08-15T17:57:34.080 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375-g2c991c19-1focal 2024-08-15T17:57:34.080 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375-g2c991c19-1focal is installed. 2024-08-15T17:57:34.082 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T17:57:34.228 DEBUG:teuthology.orchestra.run.smithi104:> dpkg-query -W -f '${Version}' ceph 2024-08-15T17:57:34.252 INFO:teuthology.orchestra.run.smithi104.stdout:17.2.7-1375-g2c991c19-1focal 2024-08-15T17:57:34.252 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375-g2c991c19-1focal 2024-08-15T17:57:34.253 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375-g2c991c19-1focal is installed. 2024-08-15T17:57:34.254 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-15T17:57:34.254 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:57:34.255 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-15T17:57:34.271 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:57:34.271 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-15T17:57:34.309 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-15T17:57:34.309 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:57:34.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-08-15T17:57:34.329 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-15T17:57:34.340 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:57:34.341 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-08-15T17:57:34.356 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-15T17:57:34.414 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-15T17:57:34.414 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:57:34.414 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-15T17:57:34.431 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-15T17:57:34.489 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:57:34.489 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-15T17:57:34.505 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-15T17:57:34.562 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-15T17:57:34.562 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:57:34.562 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-08-15T17:57:34.578 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-15T17:57:34.636 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:57:34.636 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-08-15T17:57:34.650 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-15T17:57:34.706 INFO:teuthology.run_tasks:Running task ceph... 2024-08-15T17:57:34.790 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-08-15T17:57:34.790 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /var/log/ceph 2024-08-15T17:57:34.793 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /var/log/ceph 2024-08-15T17:57:34.808 INFO:tasks.ceph:Disabling ceph logrotate... 2024-08-15T17:57:34.809 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-15T17:57:34.851 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-15T17:57:34.867 INFO:tasks.ceph:Creating extra log directories... 2024-08-15T17:57:34.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-15T17:57:34.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-15T17:57:34.885 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-08-15T17:57:34.886 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-08-15T17:57:34.886 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-08-15T17:57:34.886 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:57:34.886 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-15T17:57:34.932 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-15T17:57:34.971 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:57:34.971 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-15T17:57:34.978 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-15T17:57:35.063 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-08-15T17:57:35.063 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-08-15T17:57:35.063 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038', 'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7857038', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 25, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5995, 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549', 'targets': {'smithi038.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi104.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '67db276914d29505c9707d04f3a3b8fb9d8aee4b', 'timestamp': '2024-08-15_16:39:12', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-08-15T17:57:35.064 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-15T17:57:35.066 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-15T17:57:35.072 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m0777 -- /var/run/ceph 2024-08-15T17:57:35.115 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m0777 -- /var/run/ceph 2024-08-15T17:57:35.130 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:57:35.130 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-08-15T17:57:35.135 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-15T17:57:35.136 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-08-15T17:57:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-15T17:57:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:57:35.185 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-08-15T17:57:35.185 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:57:35.185 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-08-15 17:53:21.501148076 +0000 2024-08-15T17:57:35.185 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-08-15 17:53:21.497148169 +0000 2024-08-15T17:57:35.185 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-08-15 17:53:21.497148169 +0000 2024-08-15T17:57:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-08-15T17:57:35.186 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-15T17:57:35.200 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-08-15T17:57:35.200 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-08-15T17:57:35.201 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000499143 s, 1.0 MB/s 2024-08-15T17:57:35.201 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-15T17:57:35.210 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-08-15T17:57:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-15T17:57:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:57:35.265 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 366 Links: 1 2024-08-15T17:57:35.265 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:57:35.265 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-08-15 17:53:22.137133410 +0000 2024-08-15T17:57:35.265 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-08-15 17:53:21.929138207 +0000 2024-08-15T17:57:35.265 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-08-15 17:53:21.929138207 +0000 2024-08-15T17:57:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-08-15T17:57:35.265 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-15T17:57:35.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-08-15T17:57:35.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-08-15T17:57:35.279 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000492516 s, 1.0 MB/s 2024-08-15T17:57:35.280 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-15T17:57:35.288 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-08-15T17:57:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-15T17:57:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:57:35.293 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-08-15T17:57:35.294 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:57:35.294 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-08-15 17:53:22.593122894 +0000 2024-08-15T17:57:35.294 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-08-15 17:53:22.381127782 +0000 2024-08-15T17:57:35.294 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-08-15 17:53:22.381127782 +0000 2024-08-15T17:57:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-08-15T17:57:35.294 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-15T17:57:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-08-15T17:57:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-08-15T17:57:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000495449 s, 1.0 MB/s 2024-08-15T17:57:35.307 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-15T17:57:35.358 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-08-15T17:57:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-15T17:57:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:57:35.365 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 380 Links: 1 2024-08-15T17:57:35.365 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:57:35.365 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-08-15 17:53:23.009113300 +0000 2024-08-15T17:57:35.365 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-08-15 17:53:22.789118373 +0000 2024-08-15T17:57:35.365 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-08-15 17:53:22.789118373 +0000 2024-08-15T17:57:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-08-15T17:57:35.366 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-15T17:57:35.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-08-15T17:57:35.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-08-15T17:57:35.378 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000550408 s, 930 kB/s 2024-08-15T17:57:35.380 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-15T17:57:35.430 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T17:57:35.430 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:57:35.430 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-08-15T17:57:35.438 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-15T17:57:35.438 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-08-15T17:57:35.445 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-15T17:57:35.445 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:57:35.445 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 358 Links: 1 2024-08-15T17:57:35.446 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:57:35.446 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-08-15 17:53:20.000767348 +0000 2024-08-15T17:57:35.446 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-08-15 17:53:19.792772140 +0000 2024-08-15T17:57:35.446 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-08-15 17:53:19.792772140 +0000 2024-08-15T17:57:35.446 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2024-08-15T17:57:35.446 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-15T17:57:35.458 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-08-15T17:57:35.459 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-08-15T17:57:35.459 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000421877 s, 1.2 MB/s 2024-08-15T17:57:35.460 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-15T17:57:35.469 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-08-15T17:57:35.477 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-15T17:57:35.477 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:57:35.477 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-08-15T17:57:35.477 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:57:35.478 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-08-15 17:53:20.464756658 +0000 2024-08-15T17:57:35.478 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-08-15 17:53:20.240761819 +0000 2024-08-15T17:57:35.478 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-08-15 17:53:20.240761819 +0000 2024-08-15T17:57:35.478 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2024-08-15T17:57:35.478 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-15T17:57:35.489 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-08-15T17:57:35.489 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-08-15T17:57:35.489 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000420594 s, 1.2 MB/s 2024-08-15T17:57:35.490 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-15T17:57:35.540 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-08-15T17:57:35.548 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-15T17:57:35.548 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:57:35.548 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 369 Links: 1 2024-08-15T17:57:35.548 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:57:35.548 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-08-15 17:53:20.908746430 +0000 2024-08-15T17:57:35.548 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-08-15 17:53:20.696751314 +0000 2024-08-15T17:57:35.548 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-08-15 17:53:20.696751314 +0000 2024-08-15T17:57:35.548 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2024-08-15T17:57:35.549 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-15T17:57:35.560 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-08-15T17:57:35.561 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-08-15T17:57:35.561 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000343967 s, 1.5 MB/s 2024-08-15T17:57:35.562 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-15T17:57:35.612 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-08-15T17:57:35.619 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-15T17:57:35.619 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:57:35.619 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-08-15T17:57:35.620 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:57:35.620 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-08-15 17:53:21.332736663 +0000 2024-08-15T17:57:35.620 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-08-15 17:53:21.136741178 +0000 2024-08-15T17:57:35.620 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-08-15 17:53:21.136741178 +0000 2024-08-15T17:57:35.620 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2024-08-15T17:57:35.620 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-15T17:57:35.632 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-08-15T17:57:35.632 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-08-15T17:57:35.632 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000313207 s, 1.6 MB/s 2024-08-15T17:57:35.635 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-15T17:57:35.644 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T17:57:35.644 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi104.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-15T17:57:35.644 INFO:tasks.ceph:Generating config... 2024-08-15T17:57:35.645 INFO:tasks.ceph:[client] client mount timeout = 600 2024-08-15T17:57:35.646 INFO:tasks.ceph:[client] debug client = 20 2024-08-15T17:57:35.646 INFO:tasks.ceph:[client] debug ms = 1 2024-08-15T17:57:35.646 INFO:tasks.ceph:[client] pid file = 2024-08-15T17:57:35.646 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-08-15T17:57:35.646 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-08-15T17:57:35.646 INFO:tasks.ceph:[mds] debug mds = 20 2024-08-15T17:57:35.646 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-08-15T17:57:35.646 INFO:tasks.ceph:[mds] debug ms = 1 2024-08-15T17:57:35.646 INFO:tasks.ceph:[mds] mds debug frag = True 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mgr] debug client = 20 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mgr] debug ms = 1 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mon] debug mon = 20 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mon] debug ms = 1 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mon] debug paxos = 20 2024-08-15T17:57:35.647 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] bdev async discard = True 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] debug ms = 1 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] debug osd = 20 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-08-15T17:57:35.648 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-08-15T17:57:35.649 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-08-15T17:57:35.649 INFO:tasks.ceph:Setting up mon.a... 2024-08-15T17:57:35.649 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-08-15T17:57:35.682 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.keyring 2024-08-15T17:57:35.685 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-08-15T17:57:35.721 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-15T17:57:35.777 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.38'), ('mon.b', '172.21.15.104'), ('mon.c', '[v2:172.21.15.104:3301,v1:172.21.15.104:6790]')] 2024-08-15T17:57:35.778 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.38,172.21.15.104,[v2:172.21.15.104:3301,v1:172.21.15.104:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-08-15T17:57:35.779 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:57:35.779 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-08-15T17:57:35.787 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.38 --add b 172.21.15.104 --addv c '[v2:172.21.15.104:3301,v1:172.21.15.104:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:57:35.826 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:57:35.826 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: generated fsid 3c800201-a1b3-4c6d-887f-37b16c41eb0e 2024-08-15T17:57:35.826 INFO:teuthology.orchestra.run.smithi038.stdout:setting min_mon_release = octopus 2024-08-15T17:57:35.826 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 0 2024-08-15T17:57:35.827 INFO:teuthology.orchestra.run.smithi038.stdout:fsid 3c800201-a1b3-4c6d-887f-37b16c41eb0e 2024-08-15T17:57:35.827 INFO:teuthology.orchestra.run.smithi038.stdout:last_changed 2024-08-15T17:57:35.829137+0000 2024-08-15T17:57:35.827 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-08-15T17:57:35.829137+0000 2024-08-15T17:57:35.827 INFO:teuthology.orchestra.run.smithi038.stdout:min_mon_release 15 (octopus) 2024-08-15T17:57:35.827 INFO:teuthology.orchestra.run.smithi038.stdout:election_strategy: 1 2024-08-15T17:57:35.827 INFO:teuthology.orchestra.run.smithi038.stdout:0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-08-15T17:57:35.827 INFO:teuthology.orchestra.run.smithi038.stdout:1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-08-15T17:57:35.828 INFO:teuthology.orchestra.run.smithi038.stdout:2: [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] mon.c 2024-08-15T17:57:35.828 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-08-15T17:57:35.829 DEBUG:teuthology.orchestra.run.smithi038:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-08-15T17:57:35.876 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3c800201-a1b3-4c6d-887f-37b16c41eb0e... 2024-08-15T17:57:35.877 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-15T17:57:35.879 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-15T17:57:35.910 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2024-08-15T17:57:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: chdir = "" 2024-08-15T17:57:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-15T17:57:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: auth supported = cephx 2024-08-15T17:57:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: filestore xattr use omap = true 2024-08-15T17:57:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: mon clock drift allowed = 1.000 2024-08-15T17:57:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: osd crush chooseleaf type = 0 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: auth debug = true 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: ms die on old message = true 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: ms die on bug = true 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: mon pg warn max object skew = 0 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: # disable pg_autoscaler by default for new pools 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: osd pool default size = 2 2024-08-15T17:57:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd allow primary affinity = true 2024-08-15T17:57:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd allow pg remap = true 2024-08-15T17:57:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on legacy crush tunables = false 2024-08-15T17:57:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on crush straw calc version zero = false 2024-08-15T17:57:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on no sortbitwise = false 2024-08-15T17:57:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on osd down out interval zero = false 2024-08-15T17:57:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on too few osds = false 2024-08-15T17:57:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-15T17:57:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-15T17:57:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: mon_allow_pool_size_one = true 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: osd default data pool replay window = 5 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: mon allow pool delete = true 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: mon cluster log file level = debug 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: debug asserts on shutdown = true 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: mon health detail to clog = false 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: mon host = "172.21.15.38,172.21.15.104,[v2:172.21.15.104:3301,v1:172.21.15.104:6790]" 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = 3c800201-a1b3-4c6d-887f-37b16c41eb0e 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.913 INFO:teuthology.orchestra.run.smithi038.stdout:[osd] 2024-08-15T17:57:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: osd journal size = 100 2024-08-15T17:57:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: osd scrub load threshold = 5.0 2024-08-15T17:57:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: osd scrub max interval = 600 2024-08-15T17:57:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: osd mclock profile = high_recovery_ops 2024-08-15T17:57:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: osd recover clone overlap = true 2024-08-15T17:57:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: osd recovery max chunk = 1048576 2024-08-15T17:57:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug shutdown = true 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug op order = true 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug verify stray on activate = true 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: osd open classes on start = true 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug pg log writeout = true 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: osd deep scrub update digest min age = 30 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: osd map max advance = 10 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: journal zero on create = true 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: filestore ondisk finisher threads = 3 2024-08-15T17:57:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: filestore apply finisher threads = 3 2024-08-15T17:57:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: bdev debug aio = true 2024-08-15T17:57:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug misdirected ops = true 2024-08-15T17:57:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: bdev async discard = True 2024-08-15T17:57:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: bdev enable discard = True 2024-08-15T17:57:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: bluestore allocator = bitmap 2024-08-15T17:57:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: bluestore block size = 96636764160 2024-08-15T17:57:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: bluestore fsck on mount = True 2024-08-15T17:57:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: debug bluefs = 1/20 2024-08-15T17:57:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: debug bluestore = 1/20 2024-08-15T17:57:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-08-15T17:57:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: debug osd = 20 2024-08-15T17:57:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: debug rocksdb = 4/10 2024-08-15T17:57:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd backfillfull_ratio = 0.85 2024-08-15T17:57:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd full ratio = 0.9 2024-08-15T17:57:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd nearfull ratio = 0.8 2024-08-15T17:57:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: osd failsafe full ratio = 0.95 2024-08-15T17:57:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: osd objectstore = bluestore 2024-08-15T17:57:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: osd op complaint time = 180 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr] 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: debug mgr = 20 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: debug mon = 20 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: debug auth = 20 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min pgs per osd = 4 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min bytes per osd = 10 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: mgr/telemetry/nag = false 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: debug client = 20 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: debug finisher = 20 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout:[mon] 2024-08-15T17:57:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: debug mon = 20 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: debug paxos = 20 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: debug auth = 20 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: mon data avail warn = 5 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: mon mgr mkfs grace = 240 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min pgs per osd = 4 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd reporter subtree level = osd 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd prime pg temp = true 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min bytes per osd = 10 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: auth mon ticket ttl = 660 # 11m 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: auth service ticket ttl = 240 # 4m 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: # don't complain about insecure global_id in the test suite 2024-08-15T17:57:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-15T17:57:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-15T17:57:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: # 1m isn't quite enough 2024-08-15T17:57:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: mon_down_mkfs_grace = 2m 2024-08-15T17:57:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_filestore_osds = false 2024-08-15T17:57:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: mon op complaint time = 120 2024-08-15T17:57:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:57:35.920 INFO:teuthology.orchestra.run.smithi038.stdout:[client] 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: rgw cache enabled = true 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: rgw enable ops log = true 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: rgw enable usage log = true 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: client mount timeout = 600 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: debug client = 20 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: pid file = "" 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: rados mon op timeout = 900 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: rados osd op timeout = 900 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.a] 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.b] 2024-08-15T17:57:35.921 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.c] 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a] 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c] 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b] 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.d] 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout:[mds] 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: debug mds = 20 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: debug mds balancer = 20 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: mds debug frag = True 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: mds debug scatterstat = True 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: mds op complaint time = 180 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: mds verify scatter = True 2024-08-15T17:57:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: osd op complaint time = 180 2024-08-15T17:57:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: rados mon op timeout = 900 2024-08-15T17:57:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: rados osd op timeout = 900 2024-08-15T17:57:35.923 INFO:teuthology.orchestra.run.smithi104.stdout:[global] 2024-08-15T17:57:35.924 INFO:teuthology.orchestra.run.smithi104.stdout: chdir = "" 2024-08-15T17:57:35.924 INFO:teuthology.orchestra.run.smithi104.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-15T17:57:35.924 INFO:teuthology.orchestra.run.smithi104.stdout: auth supported = cephx 2024-08-15T17:57:35.924 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.924 INFO:teuthology.orchestra.run.smithi104.stdout: filestore xattr use omap = true 2024-08-15T17:57:35.924 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.924 INFO:teuthology.orchestra.run.smithi104.stdout: mon clock drift allowed = 1.000 2024-08-15T17:57:35.924 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.924 INFO:teuthology.orchestra.run.smithi104.stdout: osd crush chooseleaf type = 0 2024-08-15T17:57:35.924 INFO:teuthology.orchestra.run.smithi104.stdout: auth debug = true 2024-08-15T17:57:35.924 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: ms die on old message = true 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: ms die on bug = true 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: mon pg warn max object skew = 0 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: # disable pg_autoscaler by default for new pools 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: osd pool default size = 2 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd allow primary affinity = true 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd allow pg remap = true 2024-08-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on legacy crush tunables = false 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on crush straw calc version zero = false 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on no sortbitwise = false 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on osd down out interval zero = false 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on too few osds = false 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: mon_allow_pool_size_one = true 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: osd default data pool replay window = 5 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.926 INFO:teuthology.orchestra.run.smithi104.stdout: mon allow pool delete = true 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: mon cluster log file level = debug 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: debug asserts on shutdown = true 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: mon health detail to clog = false 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: mon host = "172.21.15.38,172.21.15.104,[v2:172.21.15.104:3301,v1:172.21.15.104:6790]" 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: fsid = 3c800201-a1b3-4c6d-887f-37b16c41eb0e 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout:[osd] 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: osd journal size = 100 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: osd scrub load threshold = 5.0 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: osd scrub max interval = 600 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: osd mclock profile = high_recovery_ops 2024-08-15T17:57:35.927 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: osd recover clone overlap = true 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: osd recovery max chunk = 1048576 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug shutdown = true 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug op order = true 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug verify stray on activate = true 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: osd open classes on start = true 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug pg log writeout = true 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: osd deep scrub update digest min age = 30 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: osd map max advance = 10 2024-08-15T17:57:35.928 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: journal zero on create = true 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: filestore ondisk finisher threads = 3 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: filestore apply finisher threads = 3 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: bdev debug aio = true 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug misdirected ops = true 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: bdev async discard = True 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: bdev enable discard = True 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: bluestore allocator = bitmap 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: bluestore block size = 96636764160 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: bluestore fsck on mount = True 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: debug bluefs = 1/20 2024-08-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi104.stdout: debug bluestore = 1/20 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: debug osd = 20 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: debug rocksdb = 4/10 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd backfillfull_ratio = 0.85 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd full ratio = 0.9 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd nearfull ratio = 0.8 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: osd failsafe full ratio = 0.95 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: osd objectstore = bluestore 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: osd op complaint time = 180 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout:[mgr] 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: debug mgr = 20 2024-08-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi104.stdout: debug mon = 20 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: debug auth = 20 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min pgs per osd = 4 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min bytes per osd = 10 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: mgr/telemetry/nag = false 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: debug client = 20 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: debug finisher = 20 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout:[mon] 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: debug mon = 20 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: debug paxos = 20 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: debug auth = 20 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: mon data avail warn = 5 2024-08-15T17:57:35.931 INFO:teuthology.orchestra.run.smithi104.stdout: mon mgr mkfs grace = 240 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min pgs per osd = 4 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd reporter subtree level = osd 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd prime pg temp = true 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min bytes per osd = 10 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: auth mon ticket ttl = 660 # 11m 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: auth service ticket ttl = 240 # 4m 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: # don't complain about insecure global_id in the test suite 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.932 INFO:teuthology.orchestra.run.smithi104.stdout: # 1m isn't quite enough 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: mon_down_mkfs_grace = 2m 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_filestore_osds = false 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: mon op complaint time = 120 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout:[client] 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: rgw cache enabled = true 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: rgw enable ops log = true 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: rgw enable usage log = true 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: client mount timeout = 600 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: debug client = 20 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-08-15T17:57:35.933 INFO:teuthology.orchestra.run.smithi104.stdout: pid file = "" 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout: rados mon op timeout = 900 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout: rados osd op timeout = 900 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout:[mon.a] 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout:[mon.b] 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout:[mon.c] 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.a] 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.c] 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.b] 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.d] 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout:[mds] 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout: debug mds = 20 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout: debug mds balancer = 20 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-08-15T17:57:35.934 INFO:teuthology.orchestra.run.smithi104.stdout: mds debug frag = True 2024-08-15T17:57:35.935 INFO:teuthology.orchestra.run.smithi104.stdout: mds debug scatterstat = True 2024-08-15T17:57:35.935 INFO:teuthology.orchestra.run.smithi104.stdout: mds op complaint time = 180 2024-08-15T17:57:35.935 INFO:teuthology.orchestra.run.smithi104.stdout: mds verify scatter = True 2024-08-15T17:57:35.935 INFO:teuthology.orchestra.run.smithi104.stdout: osd op complaint time = 180 2024-08-15T17:57:35.935 INFO:teuthology.orchestra.run.smithi104.stdout: rados mon op timeout = 900 2024-08-15T17:57:35.935 INFO:teuthology.orchestra.run.smithi104.stdout: rados osd op timeout = 900 2024-08-15T17:57:35.936 INFO:tasks.ceph:Creating admin key on mon.a... 2024-08-15T17:57:35.936 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-08-15T17:57:36.027 INFO:tasks.ceph:Copying monmap to all nodes... 2024-08-15T17:57:36.028 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:57:36.028 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-08-15T17:57:36.034 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:57:36.035 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-08-15T17:57:36.041 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T17:57:36.041 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:57:36.041 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-15T17:57:36.041 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-15T17:57:36.067 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:57:36.067 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:57:36.116 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T17:57:36.116 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:57:36.116 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-15T17:57:36.116 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-15T17:57:36.144 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:57:36.144 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:57:36.193 INFO:tasks.ceph:Setting up mon nodes... 2024-08-15T17:57:36.193 INFO:tasks.ceph:Setting up mgr nodes... 2024-08-15T17:57:36.194 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-08-15T17:57:36.240 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-08-15T17:57:36.242 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-08-15T17:57:36.289 INFO:teuthology.orchestra.run.smithi104.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-08-15T17:57:36.291 INFO:tasks.ceph:Setting up mds nodes... 2024-08-15T17:57:36.291 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-08-15T17:57:36.338 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-08-15T17:57:36.341 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-08-15T17:57:36.355 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-08-15T17:57:36.396 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-08-15T17:57:36.399 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-08-15T17:57:36.414 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-08-15T17:57:36.462 INFO:teuthology.orchestra.run.smithi104.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-08-15T17:57:36.464 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-08-15T17:57:36.474 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-08-15T17:57:36.515 INFO:teuthology.orchestra.run.smithi104.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-08-15T17:57:36.517 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-08-15T17:57:36.531 INFO:tasks.ceph_client:Setting up client nodes... 2024-08-15T17:57:36.531 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-15T17:57:36.569 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-08-15T17:57:36.580 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-08-15T17:57:36.611 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-08-15T17:57:36.623 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-08-15T17:57:36.660 INFO:teuthology.orchestra.run.smithi104.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-08-15T17:57:36.674 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-08-15T17:57:36.750 INFO:teuthology.orchestra.run.smithi104.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-08-15T17:57:36.763 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-08-15T17:57:36.763 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi104.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-15T17:57:36.763 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-08-15T17:57:36.779 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T17:57:36.779 INFO:tasks.ceph:role: osd.0 2024-08-15T17:57:36.779 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T17:57:36.779 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-15T17:57:38.187 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:57:38.187 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:57:38.187 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:57:38.187 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 2024-08-15T17:57:38.187 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:57:38.187 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:57:38.187 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:57:38.187 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:57:38.187 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:57:38.188 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:57:38.189 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-08-15T17:57:38.189 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-08-15T17:57:38.214 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-08-15T17:57:38.226 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:57:38.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:57:38.227 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-08-15T17:57:38.289 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T17:57:38.289 INFO:tasks.ceph:role: osd.1 2024-08-15T17:57:38.289 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T17:57:38.290 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-15T17:57:39.646 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:57:39.646 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:57:39.647 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:57:39.647 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 2024-08-15T17:57:39.647 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:57:39.647 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:57:39.647 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:57:39.647 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:57:39.647 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:57:39.647 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:57:39.648 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-08-15T17:57:39.648 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-08-15T17:57:39.675 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-08-15T17:57:39.729 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:57:39.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:57:39.730 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-08-15T17:57:39.740 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T17:57:39.740 INFO:tasks.ceph:role: osd.2 2024-08-15T17:57:39.740 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T17:57:39.741 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 2024-08-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:57:41.098 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-08-15T17:57:41.098 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-08-15T17:57:41.121 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-08-15T17:57:41.174 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:57:41.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:57:41.175 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-08-15T17:57:41.186 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T17:57:41.186 INFO:tasks.ceph:role: osd.3 2024-08-15T17:57:41.186 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T17:57:41.186 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-15T17:57:42.545 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:57:42.545 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:57:42.545 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:57:42.545 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 2024-08-15T17:57:42.545 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:57:42.545 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:57:42.545 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:57:42.546 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:57:42.546 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:57:42.546 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:57:42.547 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-08-15T17:57:42.547 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-08-15T17:57:42.571 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-08-15T17:57:42.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:57:42.626 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:57:42.627 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:57:42.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:42.691+0000 7fc5cc35c3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-08-15T17:57:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:42.691+0000 7fc5cc35c3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-08-15T17:57:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:42.691+0000 7fc5cc35c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-15T17:57:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:42.691+0000 7fc5cc35c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-15T17:57:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:42.691+0000 7fc5cc35c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-15T17:57:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:42.691+0000 7fc5cc35c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-08-15T17:57:43.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:43.251+0000 7fc5cc35c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:44.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:44.039+0000 7fc5cc35c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:44.819+0000 7fc5cc35c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:45.699 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-08-15T17:57:45.714 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:57:45.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:45.795+0000 7fd72d9a23c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-08-15T17:57:45.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:45.799+0000 7fd72d9a23c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-08-15T17:57:45.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:45.799+0000 7fd72d9a23c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-15T17:57:45.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:45.799+0000 7fd72d9a23c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-15T17:57:45.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:45.799+0000 7fd72d9a23c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-15T17:57:45.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:45.799+0000 7fd72d9a23c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-08-15T17:57:46.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:46.359+0000 7fd72d9a23c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:47.151+0000 7fd72d9a23c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:47.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:47.939+0000 7fd72d9a23c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:48.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-08-15T17:57:48.818 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:57:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:48.859+0000 7ff7c392a3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-08-15T17:57:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:48.859+0000 7ff7c392a3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-08-15T17:57:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:48.859+0000 7ff7c392a3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-15T17:57:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:48.859+0000 7ff7c392a3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-15T17:57:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:48.859+0000 7ff7c392a3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-15T17:57:48.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:48.859+0000 7ff7c392a3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-08-15T17:57:49.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:49.419+0000 7ff7c392a3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:50.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:50.215+0000 7ff7c392a3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:51.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:51.031+0000 7ff7c392a3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:51.886 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-08-15T17:57:51.946 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:57:51.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:51.983+0000 7ff07e7343c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-08-15T17:57:51.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:51.983+0000 7ff07e7343c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-08-15T17:57:51.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:51.983+0000 7ff07e7343c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-15T17:57:51.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:51.983+0000 7ff07e7343c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-15T17:57:51.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:51.983+0000 7ff07e7343c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-15T17:57:51.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:51.983+0000 7ff07e7343c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-08-15T17:57:52.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:52.547+0000 7ff07e7343c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:53.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:53.331+0000 7ff07e7343c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:54.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:57:54.119+0000 7ff07e7343c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:55.025 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-08-15T17:57:55.042 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-08-15T17:57:55.060 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T17:57:55.060 INFO:tasks.ceph:role: osd.4 2024-08-15T17:57:55.060 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T17:57:55.060 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-15T17:57:55.631 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:57:55.631 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:57:55.631 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:57:55.631 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 2024-08-15T17:57:55.631 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:57:55.631 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:57:55.631 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:57:55.631 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:57:55.632 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:57:55.632 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:57:55.632 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2024-08-15T17:57:55.632 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-08-15T17:57:55.706 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-08-15T17:57:55.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:57:55.718 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:57:55.718 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-08-15T17:57:55.776 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T17:57:55.776 INFO:tasks.ceph:role: osd.5 2024-08-15T17:57:55.776 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T17:57:55.776 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-15T17:57:56.292 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:57:56.293 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:57:56.293 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:57:56.293 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 2024-08-15T17:57:56.293 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:57:56.293 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:57:56.294 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:57:56.294 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:57:56.294 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:57:56.294 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:57:56.295 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2024-08-15T17:57:56.295 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-08-15T17:57:56.331 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-08-15T17:57:56.384 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:57:56.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:57:56.385 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-08-15T17:57:56.396 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T17:57:56.396 INFO:tasks.ceph:role: osd.6 2024-08-15T17:57:56.397 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T17:57:56.397 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-15T17:57:56.863 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:57:56.863 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:57:56.863 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:57:56.863 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 2024-08-15T17:57:56.863 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:57:56.863 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:57:56.863 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:57:56.863 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:57:56.863 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:57:56.864 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:57:56.864 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2024-08-15T17:57:56.865 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-08-15T17:57:56.912 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-08-15T17:57:56.923 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:57:56.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:57:56.924 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-08-15T17:57:56.983 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T17:57:56.983 INFO:tasks.ceph:role: osd.7 2024-08-15T17:57:56.983 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T17:57:56.983 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-15T17:57:57.477 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:57:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:57:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:57:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 2024-08-15T17:57:57.478 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:57:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:57:57.478 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:57:57.478 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:57:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:57:57.479 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:57:57.480 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2024-08-15T17:57:57.480 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-08-15T17:57:57.553 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-08-15T17:57:57.565 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:57:57.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:57:57.566 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:57:57.641 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:57:57.635+0000 7f247ca183c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-08-15T17:57:57.644 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:57:57.635+0000 7f247ca183c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-08-15T17:57:57.644 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:57:57.635+0000 7f247ca183c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-15T17:57:57.644 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:57:57.635+0000 7f247ca183c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-15T17:57:57.644 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:57:57.635+0000 7f247ca183c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-15T17:57:57.644 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:57:57.639+0000 7f247ca183c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-08-15T17:57:58.214 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:57:58.207+0000 7f247ca183c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:59.003 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:57:58.995+0000 7f247ca183c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:57:59.790 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:57:59.782+0000 7f247ca183c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:58:00.629 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-08-15T17:58:00.646 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:58:00.680 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:00.674+0000 7f764cb753c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-08-15T17:58:00.680 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:00.674+0000 7f764cb753c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-08-15T17:58:00.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:00.674+0000 7f764cb753c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-15T17:58:00.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:00.674+0000 7f764cb753c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-15T17:58:00.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:00.674+0000 7f764cb753c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-15T17:58:00.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:00.674+0000 7f764cb753c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-08-15T17:58:01.243 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:01.238+0000 7f764cb753c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:58:02.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:02.018+0000 7f764cb753c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:58:02.812 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:02.806+0000 7f764cb753c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:58:03.676 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-08-15T17:58:03.690 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:58:03.774 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:03.770+0000 7f759d62b3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-08-15T17:58:03.775 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:03.770+0000 7f759d62b3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-08-15T17:58:03.775 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:03.770+0000 7f759d62b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-15T17:58:03.775 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:03.770+0000 7f759d62b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-15T17:58:03.775 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:03.770+0000 7f759d62b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-15T17:58:03.775 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:03.770+0000 7f759d62b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-08-15T17:58:04.333 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:04.330+0000 7f759d62b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:58:05.065 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:58:05.067 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:58:05.136 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:05.134+0000 7f759d62b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:58:05.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:05.914+0000 7f759d62b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:58:06.776 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-08-15T17:58:06.792 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:58:06.875 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:06.870+0000 7f81c49313c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-08-15T17:58:06.876 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:06.870+0000 7f81c49313c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-08-15T17:58:06.876 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:06.870+0000 7f81c49313c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-15T17:58:06.876 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:06.870+0000 7f81c49313c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-15T17:58:06.876 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:06.870+0000 7f81c49313c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-15T17:58:06.876 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:06.874+0000 7f81c49313c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-08-15T17:58:07.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:07.430+0000 7f81c49313c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:58:08.214 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:08.210+0000 7f81c49313c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:58:08.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:08.990+0000 7f81c49313c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:58:09.844 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-08-15T17:58:09.860 INFO:tasks.ceph:Reading keys from all nodes... 2024-08-15T17:58:09.861 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:09.861 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-08-15T17:58:09.877 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:09.878 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-08-15T17:58:09.887 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:09.888 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-08-15T17:58:09.946 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:09.946 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-08-15T17:58:09.956 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:09.956 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-08-15T17:58:09.972 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:09.973 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-08-15T17:58:09.984 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:09.985 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-08-15T17:58:10.043 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:10.043 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-08-15T17:58:10.059 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:10.059 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-08-15T17:58:10.115 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:10.116 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-08-15T17:58:10.127 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:10.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-08-15T17:58:10.138 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:10.138 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-08-15T17:58:10.194 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:10.194 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-08-15T17:58:10.210 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:10.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-08-15T17:58:10.225 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:10.225 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-08-15T17:58:10.232 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:10.232 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-08-15T17:58:10.239 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:10.239 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-08-15T17:58:10.274 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:10.275 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-08-15T17:58:10.281 INFO:tasks.ceph:Adding keys to all mons... 2024-08-15T17:58:10.281 DEBUG:teuthology.orchestra.run.smithi038:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-15T17:58:10.284 DEBUG:teuthology.orchestra.run.smithi104:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-15T17:58:10.294 INFO:teuthology.orchestra.run.smithi104.stdout:[mgr.y] 2024-08-15T17:58:10.294 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCQQb5mM4CADhAAJobca2kE4NVAbDLtSSirRA== 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.a] 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCQQb5mQfBdFBAA3Gx7jZsEWeDgeuSMICyUmg== 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.c] 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCQQb5m9LPaFxAAY+1BXg+8eqRcz8RPF4rUbg== 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.0] 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCWQb5miglkKRAAsM9wIrp5ZtlRKFgGU89dCw== 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.1] 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCZQb5msR7DLxAA6uE5hCQk96el7O8p23jaww== 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.2] 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCcQb5mW8NfMxAAfzKINNnvlOzcuKyElEvdrg== 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.3] 2024-08-15T17:58:10.295 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCfQb5m7tbDOhAARcPt7vm+Od/j04RTgRGtqw== 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout:[mgr.x] 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCQQb5m5gIwERAA9K1VvnzPEQJpXRpnB2cy0A== 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.b] 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCQQb5msU9/GxAAtm3HTiBfvo2KZQ7FOe+yIw== 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.d] 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCQQb5msSGnHhAAciT8hAd6xi7CnYIg1pG8BQ== 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.4] 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQClQb5m71UsJhAAY2vpsr91Ms4feZhJmc9CGg== 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.5] 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCoQb5m+Rh1KBAAgryRu/S00b+kD2JdxOIKXQ== 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.6] 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCrQb5meR8PLhAAJ/OmtUi8Nze2i8FTNY6vyA== 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.7] 2024-08-15T17:58:10.296 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCuQb5mkpgTNBAAOx5A3OVlKXKjs20B8PN7VQ== 2024-08-15T17:58:10.297 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2024-08-15T17:58:10.297 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCQQb5mxxgDIhAAQrJS5uiUSE+SwUxjJDxnwQ== 2024-08-15T17:58:10.297 INFO:teuthology.orchestra.run.smithi104.stdout:[client.2] 2024-08-15T17:58:10.297 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCQQb5mLzKqJBAAZRcwc9MGtYs4OniLCvaUkg== 2024-08-15T17:58:10.297 INFO:teuthology.orchestra.run.smithi104.stdout:[client.1] 2024-08-15T17:58:10.297 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCQQb5mQQJZJxAArdQLqV7aYscbAcEaJ4sKnA== 2024-08-15T17:58:10.297 INFO:teuthology.orchestra.run.smithi104.stdout:[client.3] 2024-08-15T17:58:10.297 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCQQb5m4HirLBAAeXbg/6a1K3aCYTccAbHcwA== 2024-08-15T17:58:10.298 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr.y] 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCQQb5mM4CADhAAJobca2kE4NVAbDLtSSirRA== 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a] 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCQQb5mQfBdFBAA3Gx7jZsEWeDgeuSMICyUmg== 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c] 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCQQb5m9LPaFxAAY+1BXg+8eqRcz8RPF4rUbg== 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.0] 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCWQb5miglkKRAAsM9wIrp5ZtlRKFgGU89dCw== 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.1] 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCZQb5msR7DLxAA6uE5hCQk96el7O8p23jaww== 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.2] 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCcQb5mW8NfMxAAfzKINNnvlOzcuKyElEvdrg== 2024-08-15T17:58:10.299 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.3] 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCfQb5m7tbDOhAARcPt7vm+Od/j04RTgRGtqw== 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr.x] 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCQQb5m5gIwERAA9K1VvnzPEQJpXRpnB2cy0A== 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b] 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCQQb5msU9/GxAAtm3HTiBfvo2KZQ7FOe+yIw== 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.d] 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCQQb5msSGnHhAAciT8hAd6xi7CnYIg1pG8BQ== 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.4] 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQClQb5m71UsJhAAY2vpsr91Ms4feZhJmc9CGg== 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.5] 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCoQb5m+Rh1KBAAgryRu/S00b+kD2JdxOIKXQ== 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.6] 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCrQb5meR8PLhAAJ/OmtUi8Nze2i8FTNY6vyA== 2024-08-15T17:58:10.300 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.7] 2024-08-15T17:58:10.301 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCuQb5mkpgTNBAAOx5A3OVlKXKjs20B8PN7VQ== 2024-08-15T17:58:10.301 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-08-15T17:58:10.301 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCQQb5mxxgDIhAAQrJS5uiUSE+SwUxjJDxnwQ== 2024-08-15T17:58:10.301 INFO:teuthology.orchestra.run.smithi038.stdout:[client.2] 2024-08-15T17:58:10.301 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCQQb5mLzKqJBAAZRcwc9MGtYs4OniLCvaUkg== 2024-08-15T17:58:10.301 INFO:teuthology.orchestra.run.smithi038.stdout:[client.1] 2024-08-15T17:58:10.301 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCQQb5mQQJZJxAArdQLqV7aYscbAcEaJ4sKnA== 2024-08-15T17:58:10.301 INFO:teuthology.orchestra.run.smithi038.stdout:[client.3] 2024-08-15T17:58:10.301 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCQQb5m4HirLBAAeXbg/6a1K3aCYTccAbHcwA== 2024-08-15T17:58:10.301 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T17:58:10.304 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T17:58:10.344 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:58:10.388 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:58:10.437 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:58:10.440 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:58:10.486 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:10.488 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:10.529 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:10.572 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:10.618 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:10.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:10.664 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:10.667 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:10.752 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T17:58:10.754 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T17:58:10.799 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:58:10.801 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:58:10.847 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:58:10.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:58:10.934 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:10.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:10.982 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:10.984 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:11.028 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:11.031 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:11.116 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:11.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:58:11.164 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:58:11.166 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:58:11.213 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:58:11.214 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:58:11.303 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:58:11.306 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:58:11.352 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:58:11.354 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:58:11.402 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-08-15T17:58:11.402 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-08-15T17:58:11.418 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-15T17:58:11.496 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-08-15T17:58:11.513 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-08-15T17:58:11.530 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-15T17:58:11.583 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-08-15T17:58:11.597 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-08-15T17:58:11.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-15T17:58:11.711 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-08-15T17:58:11.725 DEBUG:teuthology.orchestra.run.smithi038:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:58:11.727 DEBUG:teuthology.orchestra.run.smithi104:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:58:11.731 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-08-15T17:58:11.731 INFO:tasks.ceph.mon.a:Restarting daemon 2024-08-15T17:58:11.732 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-08-15T17:58:11.776 INFO:tasks.ceph.mon.a:Started 2024-08-15T17:58:11.776 INFO:tasks.ceph.mon.b:Restarting daemon 2024-08-15T17:58:11.777 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-08-15T17:58:11.779 INFO:tasks.ceph.mon.b:Started 2024-08-15T17:58:11.780 INFO:tasks.ceph.mon.c:Restarting daemon 2024-08-15T17:58:11.780 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-08-15T17:58:11.782 INFO:tasks.ceph.mon.c:Started 2024-08-15T17:58:11.782 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-08-15T17:58:11.782 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-08-15T17:58:11.782 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-08-15T17:58:11.785 INFO:tasks.ceph.mgr.y:Started 2024-08-15T17:58:11.785 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-08-15T17:58:11.786 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-08-15T17:58:11.788 INFO:tasks.ceph.mgr.x:Started 2024-08-15T17:58:11.788 INFO:tasks.ceph:Setting crush tunables to default 2024-08-15T17:58:11.788 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd crush tunables default 2024-08-15T17:58:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.883+0000 7f7d477fe700 1 Processor -- start 2024-08-15T17:58:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.883+0000 7f7d477fe700 1 -- start start 2024-08-15T17:58:11.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.883+0000 7f7d477fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7d400a9520 0x7f7d4006d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:11.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.883+0000 7f7d477fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7d4006daa0 0x7f7d400af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.883+0000 7f7d477fe700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d400af220 con 0x7f7d40142970 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.883+0000 7f7d477fe700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4006e320 con 0x7f7d4006daa0 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.883+0000 7f7d477fe700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4006e600 con 0x7f7d400a9520 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.883+0000 7f7d45d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7d4006daa0 0x7f7d400af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.883+0000 7f7d4559a700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7d40142970 0x7f7d400a8e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33962/0 (socket says 172.21.15.38:33962) 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.883+0000 7f7d4559a700 1 -- 172.21.15.38:0/1212684952 learned_addr learned my addr 172.21.15.38:0/1212684952 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.887+0000 7f7d44d99700 1 --2- 172.21.15.38:0/1212684952 >> v2:172.21.15.104:3300/0 conn(0x7f7d400a9520 0x7f7d4006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.887+0000 7f7d45d9b700 1 -- 172.21.15.38:0/1212684952 >> v2:172.21.15.38:3300/0 conn(0x7f7d4006daa0 msgr2=0x7f7d400af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.887+0000 7f7d45d9b700 1 -- 172.21.15.38:0/1212684952 >> v2:172.21.15.38:3300/0 conn(0x7f7d4006daa0 msgr2=0x7f7d400af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.887+0000 7f7d45d9b700 1 --2- 172.21.15.38:0/1212684952 >> v2:172.21.15.38:3300/0 conn(0x7f7d4006daa0 0x7f7d400af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.887+0000 7f7d44d99700 1 -- 172.21.15.38:0/1212684952 >> v2:172.21.15.104:3300/0 conn(0x7f7d400a9520 msgr2=0x7f7d4006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.887+0000 7f7d44d99700 1 -- 172.21.15.38:0/1212684952 >> v2:172.21.15.104:3300/0 conn(0x7f7d400a9520 msgr2=0x7f7d4006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:58:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.887+0000 7f7d45d9b700 1 --2- 172.21.15.38:0/1212684952 >> v2:172.21.15.38:3300/0 conn(0x7f7d4006daa0 0x7f7d400af6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-15T17:58:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.887+0000 7f7d44d99700 1 --2- 172.21.15.38:0/1212684952 >> v2:172.21.15.104:3300/0 conn(0x7f7d400a9520 0x7f7d4006d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:58:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.887+0000 7f7d44d99700 1 --2- 172.21.15.38:0/1212684952 >> v2:172.21.15.104:3300/0 conn(0x7f7d400a9520 0x7f7d4006d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-15T17:58:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.959+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874423603 0 0) 0x7f7d400af220 con 0x7f7d40142970 2024-08-15T17:58:11.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.959+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d2c003570 con 0x7f7d40142970 2024-08-15T17:58:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.959+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 852667805 0 0) 0x7f7d2c003570 con 0x7f7d40142970 2024-08-15T17:58:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.959+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 >> v2:172.21.15.104:3300/0 conn(0x7f7d400a9520 msgr2=0x7f7d4006d560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T17:58:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.959+0000 7f7d37fff700 1 --2- 172.21.15.38:0/1212684952 >> v2:172.21.15.104:3300/0 conn(0x7f7d400a9520 0x7f7d4006d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 >> v2:172.21.15.38:3300/0 conn(0x7f7d4006daa0 msgr2=0x7f7d400af6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T17:58:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d37fff700 1 --2- 172.21.15.38:0/1212684952 >> v2:172.21.15.38:3300/0 conn(0x7f7d4006daa0 0x7f7d400af6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4006e8e0 con 0x7f7d40142970 2024-08-15T17:58:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f7d38001470 con 0x7f7d40142970 2024-08-15T17:58:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 >> v1:172.21.15.38:6789/0 conn(0x7f7d40142970 legacy=0x7f7d400a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d37fff700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d2c003a60 0x7f7d2c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d37fff700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d2c004810 0x7f7d2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d37fff700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d2c0051c0 0x7f7d2c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4006e320 con 0x7f7d2c003a60 2024-08-15T17:58:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4006e600 con 0x7f7d2c004810 2024-08-15T17:58:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c00c1b0 con 0x7f7d2c0051c0 2024-08-15T17:58:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d4559a700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d2c003a60 0x7f7d2c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d45d9b700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d2c0051c0 0x7f7d2c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:11.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d44d99700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d2c004810 0x7f7d2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:11.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d4559a700 1 -- 172.21.15.38:0/1212684952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d2c0051c0 msgr2=0x7f7d2c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:11.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d4559a700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d2c0051c0 0x7f7d2c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:11.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d4559a700 1 -- 172.21.15.38:0/1212684952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d2c004810 msgr2=0x7f7d2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:11.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d4559a700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d2c004810 0x7f7d2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:11.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d4559a700 1 -- 172.21.15.38:0/1212684952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c003570 con 0x7f7d2c003a60 2024-08-15T17:58:11.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d45d9b700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d2c0051c0 0x7f7d2c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:11.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d45d9b700 1 -- 172.21.15.38:0/1212684952 reap_dead start 2024-08-15T17:58:11.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d44d99700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d2c004810 0x7f7d2c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:11.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.963+0000 7f7d4559a700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d2c003a60 0x7f7d2c003e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c3c0 tx=0x7f7d38001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9512636c003365ac server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:11.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.967+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800cc90 con 0x7f7d2c003a60 2024-08-15T17:58:11.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.967+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800ce10 con 0x7f7d2c003a60 2024-08-15T17:58:11.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.967+0000 7f7d37fff700 1 -- 172.21.15.38:0/1212684952 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38005b70 con 0x7f7d2c003a60 2024-08-15T17:58:11.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.967+0000 7f7d477fe700 1 -- 172.21.15.38:0/1212684952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d2c003a60 msgr2=0x7f7d2c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:11.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.967+0000 7f7d477fe700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d2c003a60 0x7f7d2c003e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c3c0 tx=0x7f7d38001e90 comp rx=0 tx=0).stop 2024-08-15T17:58:11.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.967+0000 7f7d477fe700 1 -- 172.21.15.38:0/1212684952 shutdown_connections 2024-08-15T17:58:11.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.967+0000 7f7d477fe700 1 --2- 172.21.15.38:0/1212684952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d2c003a60 0x7f7d2c003e70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:11.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.967+0000 7f7d477fe700 1 -- 172.21.15.38:0/1212684952 >> 172.21.15.38:0/1212684952 conn(0x7f7d40066e00 msgr2=0x7f7d40067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:11.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.967+0000 7f7d477fe700 1 -- 172.21.15.38:0/1212684952 shutdown_connections 2024-08-15T17:58:11.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.967+0000 7f7d477fe700 1 -- 172.21.15.38:0/1212684952 wait complete. 2024-08-15T17:58:11.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d477fe700 1 Processor -- start 2024-08-15T17:58:11.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d477fe700 1 -- start start 2024-08-15T17:58:11.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d477fe700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d2c003a60 0x7f7d4016c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:11.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d477fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d40098600 0x7f7d4013e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:11.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d477fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d400abae0 0x7f7d400abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:11.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d477fe700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4015b3f0 con 0x7f7d40098600 2024-08-15T17:58:11.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d477fe700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4016d240 con 0x7f7d400abae0 2024-08-15T17:58:11.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d477fe700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4016d520 con 0x7f7d2c003a60 2024-08-15T17:58:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d44d99700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d40098600 0x7f7d4013e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d45d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d400abae0 0x7f7d400abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d4559a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d2c003a60 0x7f7d4016c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d44d99700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d40098600 0x7f7d4013e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37740/0 (socket says 172.21.15.38:37740) 2024-08-15T17:58:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d44d99700 1 -- 172.21.15.38:0/737438315 learned_addr learned my addr 172.21.15.38:0/737438315 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:11.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.971+0000 7f7d45d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d400abae0 0x7f7d400abf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:42162/0 (socket says 172.21.15.38:42162) 2024-08-15T17:58:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d44d99700 1 -- 172.21.15.38:0/737438315 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d2c003a60 msgr2=0x7f7d4016c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:11.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d44d99700 1 --2- 172.21.15.38:0/737438315 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d2c003a60 0x7f7d4016c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d44d99700 1 -- 172.21.15.38:0/737438315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d400abae0 msgr2=0x7f7d400abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d44d99700 1 --2- 172.21.15.38:0/737438315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d400abae0 0x7f7d400abf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d44d99700 1 -- 172.21.15.38:0/737438315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4016d800 con 0x7f7d40098600 2024-08-15T17:58:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d44d99700 1 --2- 172.21.15.38:0/737438315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d40098600 0x7f7d4013e0d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000f590 tx=0x7f7d30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d357fa700 1 -- 172.21.15.38:0/737438315 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000fe60 con 0x7f7d40098600 2024-08-15T17:58:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d357fa700 1 -- 172.21.15.38:0/737438315 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d30014670 con 0x7f7d40098600 2024-08-15T17:58:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d477fe700 1 -- 172.21.15.38:0/737438315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4015b080 con 0x7f7d40098600 2024-08-15T17:58:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d357fa700 1 -- 172.21.15.38:0/737438315 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30014a10 con 0x7f7d40098600 2024-08-15T17:58:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d477fe700 1 -- 172.21.15.38:0/737438315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4013e610 con 0x7f7d40098600 2024-08-15T17:58:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d357fa700 1 -- 172.21.15.38:0/737438315 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f7d30015410 con 0x7f7d40098600 2024-08-15T17:58:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d357fa700 1 -- 172.21.15.38:0/737438315 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7d30014e40 con 0x7f7d40098600 2024-08-15T17:58:11.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.975+0000 7f7d477fe700 1 -- 172.21.15.38:0/737438315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4015b560 con 0x7f7d40098600 2024-08-15T17:58:11.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:11.979+0000 7f7d357fa700 1 -- 172.21.15.38:0/737438315 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f7d3003d7a0 con 0x7f7d40098600 2024-08-15T17:58:12.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:12.063+0000 7f7d477fe700 1 -- 172.21.15.38:0/737438315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7d400645c0 con 0x7f7d40098600 2024-08-15T17:58:12.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:12.071+0000 7f7d357fa700 1 -- 172.21.15.38:0/737438315 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7d3002c020 con 0x7f7d40098600 2024-08-15T17:58:12.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:12.075+0000 7f7d477fe700 1 -- 172.21.15.38:0/737438315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d40098600 msgr2=0x7f7d4013e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:12.075+0000 7f7d477fe700 1 --2- 172.21.15.38:0/737438315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d40098600 0x7f7d4013e0d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000f590 tx=0x7f7d30008040 comp rx=0 tx=0).stop 2024-08-15T17:58:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:12.075+0000 7f7d477fe700 1 -- 172.21.15.38:0/737438315 shutdown_connections 2024-08-15T17:58:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:12.075+0000 7f7d477fe700 1 --2- 172.21.15.38:0/737438315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d400abae0 0x7f7d400abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:12.075+0000 7f7d477fe700 1 --2- 172.21.15.38:0/737438315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d40098600 0x7f7d4013e0d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:12.075+0000 7f7d477fe700 1 --2- 172.21.15.38:0/737438315 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d2c003a60 0x7f7d4016c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:12.075+0000 7f7d477fe700 1 -- 172.21.15.38:0/737438315 >> 172.21.15.38:0/737438315 conn(0x7f7d40066e00 msgr2=0x7f7d4006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:12.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:12.075+0000 7f7d477fe700 1 -- 172.21.15.38:0/737438315 shutdown_connections 2024-08-15T17:58:12.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:12.075+0000 7f7d477fe700 1 -- 172.21.15.38:0/737438315 wait complete. 2024-08-15T17:58:12.077 INFO:teuthology.orchestra.run.smithi038.stderr:adjusted tunables profile to default 2024-08-15T17:58:12.090 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-08-15T17:58:12.090 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:12.090 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-08-15T17:58:12.106 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:12.106 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-08-15T17:58:12.119 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:12.119 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-08-15T17:58:12.132 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-08-15T17:58:12.132 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-08-15T17:58:12.143 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:12.143 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-08-15T17:58:12.152 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:12.150+0000 7fb9acbbc000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-15T17:58:12.159 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:12.159 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-08-15T17:58:12.178 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:12.175+0000 7f5fa5693000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-15T17:58:12.215 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:12.215 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-08-15T17:58:12.227 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-08-15T17:58:12.228 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-08-15T17:58:12.237 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new d94c8b75-7d55-4eb9-b831-c22e47c948ad 0 2024-08-15T17:58:12.388 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.382+0000 7f4dcc85c700 1 Processor -- start 2024-08-15T17:58:12.388 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.382+0000 7f4dcc85c700 1 -- start start 2024-08-15T17:58:12.391 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4dc8158ea0 0x7f4dc8159270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.391 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 0x7f4dc80af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.391 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc80aa900 con 0x7f4dc80aea20 2024-08-15T17:58:12.391 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc8067490 con 0x7f4dc8158ea0 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc80a9cf0 con 0x7f4dc80a9860 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc759e700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f4dc80aea20 0x7f4dc80686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:33224/0 (socket says 172.21.15.104:33224) 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc759e700 1 -- 172.21.15.104:0/1498488541 learned_addr learned my addr 172.21.15.104:0/1498488541 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc659c700 1 --2- 172.21.15.104:0/1498488541 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 0x7f4dc80af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc6d9d700 1 --2- 172.21.15.104:0/1498488541 >> v2:172.21.15.38:3300/0 conn(0x7f4dc8158ea0 0x7f4dc8159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc5d9b700 1 -- 172.21.15.104:0/1498488541 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635295385 0 0) 0x7f4dc80aa900 con 0x7f4dc80aea20 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc5d9b700 1 -- 172.21.15.104:0/1498488541 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4db4003570 con 0x7f4dc80aea20 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc659c700 1 -- 172.21.15.104:0/1498488541 >> v2:172.21.15.38:3300/0 conn(0x7f4dc8158ea0 msgr2=0x7f4dc8159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc659c700 1 --2- 172.21.15.104:0/1498488541 >> v2:172.21.15.38:3300/0 conn(0x7f4dc8158ea0 0x7f4dc8159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc659c700 1 -- 172.21.15.104:0/1498488541 >> v1:172.21.15.104:6789/0 conn(0x7f4dc80aea20 legacy=0x7f4dc80686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc5d9b700 1 -- 172.21.15.104:0/1498488541 <== mon.1 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2291347326 0 0) 0x7f4db4003570 con 0x7f4dc80aea20 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc659c700 1 -- 172.21.15.104:0/1498488541 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc80a9fd0 con 0x7f4dc80a9860 2024-08-15T17:58:12.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc659c700 1 --2- 172.21.15.104:0/1498488541 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 0x7f4dc80af540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4db00098e0 tx=0x7f4db00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ef97461baad500 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:12.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc5d9b700 1 -- 172.21.15.104:0/1498488541 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0011630 con 0x7f4dc80a9860 2024-08-15T17:58:12.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc5d9b700 1 -- 172.21.15.104:0/1498488541 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db0011790 con 0x7f4dc80a9860 2024-08-15T17:58:12.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dc5d9b700 1 -- 172.21.15.104:0/1498488541 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0011b30 con 0x7f4dc80a9860 2024-08-15T17:58:12.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 -- 172.21.15.104:0/1498488541 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 msgr2=0x7f4dc80af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 --2- 172.21.15.104:0/1498488541 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 0x7f4dc80af540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4db00098e0 tx=0x7f4db00091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 -- 172.21.15.104:0/1498488541 shutdown_connections 2024-08-15T17:58:12.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 --2- 172.21.15.104:0/1498488541 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 0x7f4dc80af540 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 --2- 172.21.15.104:0/1498488541 >> v2:172.21.15.38:3300/0 conn(0x7f4dc8158ea0 0x7f4dc8159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 -- 172.21.15.104:0/1498488541 >> 172.21.15.104:0/1498488541 conn(0x7f4dc806e350 msgr2=0x7f4dc806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:12.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 -- 172.21.15.104:0/1498488541 shutdown_connections 2024-08-15T17:58:12.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.386+0000 7f4dcc85c700 1 -- 172.21.15.104:0/1498488541 wait complete. 2024-08-15T17:58:12.395 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dcc85c700 1 Processor -- start 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dcc85c700 1 -- start start 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dcc85c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 0x7f4dc80c1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dcc85c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dc80aea20 0x7f4dc815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dcc85c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dc8158ea0 0x7f4dc80c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dcc85c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc815c230 con 0x7f4dc80aea20 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dcc85c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc80c1d70 con 0x7f4dc8158ea0 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dcc85c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc80c2050 con 0x7f4dc80a9860 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc6d9d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 0x7f4dc80c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc6d9d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 0x7f4dc80c1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.104:44484/0 (socket says 172.21.15.104:44484) 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc6d9d700 1 -- 172.21.15.104:0/2683056943 learned_addr learned my addr 172.21.15.104:0/2683056943 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc759e700 1 --2- 172.21.15.104:0/2683056943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dc8158ea0 0x7f4dc80c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc659c700 1 --2- 172.21.15.104:0/2683056943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dc80aea20 0x7f4dc815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc6d9d700 1 -- 172.21.15.104:0/2683056943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dc8158ea0 msgr2=0x7f4dc80c2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc6d9d700 1 --2- 172.21.15.104:0/2683056943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dc8158ea0 0x7f4dc80c2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc6d9d700 1 -- 172.21.15.104:0/2683056943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dc80aea20 msgr2=0x7f4dc815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc6d9d700 1 --2- 172.21.15.104:0/2683056943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dc80aea20 0x7f4dc815bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc6d9d700 1 -- 172.21.15.104:0/2683056943 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db0009420 con 0x7f4dc80a9860 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc659c700 1 --2- 172.21.15.104:0/2683056943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dc80aea20 0x7f4dc815bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dc6d9d700 1 --2- 172.21.15.104:0/2683056943 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 0x7f4dc80c1670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4db800ec80 tx=0x7f4db80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dbeffd700 1 -- 172.21.15.104:0/2683056943 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db800c9d0 con 0x7f4dc80a9860 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dbeffd700 1 -- 172.21.15.104:0/2683056943 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db800cb30 con 0x7f4dc80a9860 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dcc85c700 1 -- 172.21.15.104:0/2683056943 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc80c2330 con 0x7f4dc80a9860 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dbeffd700 1 -- 172.21.15.104:0/2683056943 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8013470 con 0x7f4dc80a9860 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dcc85c700 1 -- 172.21.15.104:0/2683056943 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc8169750 con 0x7f4dc80a9860 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dbeffd700 1 -- 172.21.15.104:0/2683056943 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f4db8013820 con 0x7f4dc80a9860 2024-08-15T17:58:12.397 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dbeffd700 1 -- 172.21.15.104:0/2683056943 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4db8005db0 con 0x7f4dc80a9860 2024-08-15T17:58:12.398 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.390+0000 7f4dcc85c700 1 -- 172.21.15.104:0/2683056943 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc80638f0 con 0x7f4dc80a9860 2024-08-15T17:58:12.398 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.394+0000 7f4dbeffd700 1 -- 172.21.15.104:0/2683056943 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f4db8021060 con 0x7f4dc80a9860 2024-08-15T17:58:12.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.466+0000 7f4dcc85c700 1 -- 172.21.15.104:0/2683056943 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d94c8b75-7d55-4eb9-b831-c22e47c948ad", "id": 0} v 0) v1 -- 0x7f4dc80696a0 con 0x7f4dc80a9860 2024-08-15T17:58:12.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.478+0000 7f4dbeffd700 1 -- 172.21.15.104:0/2683056943 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d94c8b75-7d55-4eb9-b831-c22e47c948ad", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4db8017070 con 0x7f4dc80a9860 2024-08-15T17:58:12.483 INFO:teuthology.orchestra.run.smithi104.stdout:0 2024-08-15T17:58:12.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.478+0000 7f4dcc85c700 1 -- 172.21.15.104:0/2683056943 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 msgr2=0x7f4dc80c1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.478+0000 7f4dcc85c700 1 --2- 172.21.15.104:0/2683056943 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 0x7f4dc80c1670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4db800ec80 tx=0x7f4db80091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.478+0000 7f4dcc85c700 1 -- 172.21.15.104:0/2683056943 shutdown_connections 2024-08-15T17:58:12.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.478+0000 7f4dcc85c700 1 --2- 172.21.15.104:0/2683056943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dc8158ea0 0x7f4dc80c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.478+0000 7f4dcc85c700 1 --2- 172.21.15.104:0/2683056943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dc80aea20 0x7f4dc815bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.478+0000 7f4dcc85c700 1 --2- 172.21.15.104:0/2683056943 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dc80a9860 0x7f4dc80c1670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.478+0000 7f4dcc85c700 1 -- 172.21.15.104:0/2683056943 >> 172.21.15.104:0/2683056943 conn(0x7f4dc806e350 msgr2=0x7f4dc806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:12.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.478+0000 7f4dcc85c700 1 -- 172.21.15.104:0/2683056943 shutdown_connections 2024-08-15T17:58:12.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.478+0000 7f4dcc85c700 1 -- 172.21.15.104:0/2683056943 wait complete. 2024-08-15T17:58:12.499 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new bbe82cfa-6bc8-43ef-b606-7802ad47cf8e 1 2024-08-15T17:58:12.596 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.590+0000 7fc198914700 1 Processor -- start 2024-08-15T17:58:12.596 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.590+0000 7fc198914700 1 -- start start 2024-08-15T17:58:12.598 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc198914700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc19414fe10 0x7fc1940686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.598 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc198914700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc194068be0 0x7fc194155c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.598 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc198914700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1940ae120 con 0x7fc19414fa40 2024-08-15T17:58:12.598 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc198914700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc194069060 con 0x7fc19414fe10 2024-08-15T17:58:12.598 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc198914700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1941569b0 con 0x7fc194068be0 2024-08-15T17:58:12.598 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc192ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc19414fe10 0x7fc1940686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.599 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc193fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc194068be0 0x7fc194155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.599 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc192ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc19414fe10 0x7fc1940686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52920/0 (socket says 172.21.15.104:52920) 2024-08-15T17:58:12.599 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc192ffd700 1 -- 172.21.15.104:0/3762926131 learned_addr learned my addr 172.21.15.104:0/3762926131 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:12.599 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc193fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc194068be0 0x7fc194155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.104:44492/0 (socket says 172.21.15.104:44492) 2024-08-15T17:58:12.600 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc193fff700 1 -- 172.21.15.104:0/3762926131 >> v2:172.21.15.104:3300/0 conn(0x7fc19414fe10 msgr2=0x7fc1940686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:12.600 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc193fff700 1 --2- 172.21.15.104:0/3762926131 >> v2:172.21.15.104:3300/0 conn(0x7fc19414fe10 0x7fc1940686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.600 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc193fff700 1 -- 172.21.15.104:0/3762926131 >> v1:172.21.15.38:6789/0 conn(0x7fc19414fa40 legacy=0x7fc194156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.600 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc193fff700 1 -- 172.21.15.104:0/3762926131 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1940ad7f0 con 0x7fc194068be0 2024-08-15T17:58:12.600 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc192ffd700 1 --2- 172.21.15.104:0/3762926131 >> v2:172.21.15.104:3300/0 conn(0x7fc19414fe10 0x7fc1940686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:12.600 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc193fff700 1 --2- 172.21.15.104:0/3762926131 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc194068be0 0x7fc194155c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009e60 tx=0x7fc18c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adf45ec355242ea3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:12.600 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc1927fc700 1 -- 172.21.15.104:0/3762926131 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00e070 con 0x7fc194068be0 2024-08-15T17:58:12.600 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc1927fc700 1 -- 172.21.15.104:0/3762926131 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c00bb60 con 0x7fc194068be0 2024-08-15T17:58:12.600 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.594+0000 7fc1927fc700 1 -- 172.21.15.104:0/3762926131 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c012430 con 0x7fc194068be0 2024-08-15T17:58:12.601 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.598+0000 7fc198914700 1 -- 172.21.15.104:0/3762926131 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc194068be0 msgr2=0x7fc194155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.601 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.598+0000 7fc198914700 1 --2- 172.21.15.104:0/3762926131 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc194068be0 0x7fc194155c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009e60 tx=0x7fc18c009780 comp rx=0 tx=0).stop 2024-08-15T17:58:12.601 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.598+0000 7fc198914700 1 -- 172.21.15.104:0/3762926131 shutdown_connections 2024-08-15T17:58:12.601 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.598+0000 7fc198914700 1 --2- 172.21.15.104:0/3762926131 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc194068be0 0x7fc194155c80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.601 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.598+0000 7fc198914700 1 --2- 172.21.15.104:0/3762926131 >> v2:172.21.15.104:3300/0 conn(0x7fc19414fe10 0x7fc1940686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.601 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.598+0000 7fc198914700 1 -- 172.21.15.104:0/3762926131 >> 172.21.15.104:0/3762926131 conn(0x7fc19406e310 msgr2=0x7fc19406b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:12.601 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.598+0000 7fc198914700 1 -- 172.21.15.104:0/3762926131 shutdown_connections 2024-08-15T17:58:12.602 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.598+0000 7fc198914700 1 -- 172.21.15.104:0/3762926131 wait complete. 2024-08-15T17:58:12.604 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.598+0000 7fc198914700 1 Processor -- start 2024-08-15T17:58:12.604 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.598+0000 7fc198914700 1 -- start start 2024-08-15T17:58:12.605 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc198914700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc194068be0 0x7fc1941511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.605 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc198914700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc19414fa40 0x7fc1940acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.605 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc198914700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc19414fe10 0x7fc194147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.605 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc198914700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc194069d60 con 0x7fc194068be0 2024-08-15T17:58:12.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc198914700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19406a040 con 0x7fc19414fa40 2024-08-15T17:58:12.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc198914700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19406a320 con 0x7fc19414fe10 2024-08-15T17:58:12.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc193fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc19414fe10 0x7fc194147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc192ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc19414fa40 0x7fc1940acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc193fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc19414fe10 0x7fc194147570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.104:44506/0 (socket says 172.21.15.104:44506) 2024-08-15T17:58:12.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc193fff700 1 -- 172.21.15.104:0/2300513342 learned_addr learned my addr 172.21.15.104:0/2300513342 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:12.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc1937fe700 1 --2- 172.21.15.104:0/2300513342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc194068be0 0x7fc1941511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.607 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc192ffd700 1 -- 172.21.15.104:0/2300513342 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc19414fe10 msgr2=0x7fc194147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.607 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc192ffd700 1 --2- 172.21.15.104:0/2300513342 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc19414fe10 0x7fc194147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.607 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc192ffd700 1 -- 172.21.15.104:0/2300513342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc194068be0 msgr2=0x7fc1941511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.607 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc192ffd700 1 --2- 172.21.15.104:0/2300513342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc194068be0 0x7fc1941511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.607 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc192ffd700 1 -- 172.21.15.104:0/2300513342 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18800b780 con 0x7fc19414fa40 2024-08-15T17:58:12.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc192ffd700 1 --2- 172.21.15.104:0/2300513342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc19414fa40 0x7fc1940acd90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc18800dea0 tx=0x7fc18800f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:12.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc17b7fe700 1 -- 172.21.15.104:0/2300513342 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188009b00 con 0x7fc19414fa40 2024-08-15T17:58:12.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc17b7fe700 1 -- 172.21.15.104:0/2300513342 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc188009c60 con 0x7fc19414fa40 2024-08-15T17:58:12.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc198914700 1 -- 172.21.15.104:0/2300513342 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c009a20 con 0x7fc19414fa40 2024-08-15T17:58:12.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc17b7fe700 1 -- 172.21.15.104:0/2300513342 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1880173f0 con 0x7fc19414fa40 2024-08-15T17:58:12.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc198914700 1 -- 172.21.15.104:0/2300513342 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1940ad2d0 con 0x7fc19414fa40 2024-08-15T17:58:12.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc17b7fe700 1 -- 172.21.15.104:0/2300513342 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fc188005c90 con 0x7fc19414fa40 2024-08-15T17:58:12.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc17b7fe700 1 -- 172.21.15.104:0/2300513342 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fc188006020 con 0x7fc19414fa40 2024-08-15T17:58:12.609 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.602+0000 7fc198914700 1 -- 172.21.15.104:0/2300513342 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1940638f0 con 0x7fc19414fa40 2024-08-15T17:58:12.611 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.606+0000 7fc17b7fe700 1 -- 172.21.15.104:0/2300513342 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fc188017e10 con 0x7fc19414fa40 2024-08-15T17:58:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.682+0000 7fc198914700 1 -- 172.21.15.104:0/2300513342 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bbe82cfa-6bc8-43ef-b606-7802ad47cf8e", "id": 1} v 0) v1 -- 0x7fc194155f80 con 0x7fc19414fa40 2024-08-15T17:58:12.697 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.694+0000 7fc17b7fe700 1 -- 172.21.15.104:0/2300513342 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bbe82cfa-6bc8-43ef-b606-7802ad47cf8e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc1880177e0 con 0x7fc19414fa40 2024-08-15T17:58:12.698 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-08-15T17:58:12.699 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.694+0000 7fc198914700 1 -- 172.21.15.104:0/2300513342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc19414fa40 msgr2=0x7fc1940acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.699 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.694+0000 7fc198914700 1 --2- 172.21.15.104:0/2300513342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc19414fa40 0x7fc1940acd90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc18800dea0 tx=0x7fc18800f040 comp rx=0 tx=0).stop 2024-08-15T17:58:12.699 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.694+0000 7fc198914700 1 -- 172.21.15.104:0/2300513342 shutdown_connections 2024-08-15T17:58:12.699 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.694+0000 7fc198914700 1 --2- 172.21.15.104:0/2300513342 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc19414fe10 0x7fc194147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.699 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.694+0000 7fc198914700 1 --2- 172.21.15.104:0/2300513342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc19414fa40 0x7fc1940acd90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.699 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.694+0000 7fc198914700 1 --2- 172.21.15.104:0/2300513342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc194068be0 0x7fc1941511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.699 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.694+0000 7fc198914700 1 -- 172.21.15.104:0/2300513342 >> 172.21.15.104:0/2300513342 conn(0x7fc19406e310 msgr2=0x7fc1941541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:12.699 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.694+0000 7fc198914700 1 -- 172.21.15.104:0/2300513342 shutdown_connections 2024-08-15T17:58:12.699 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.694+0000 7fc198914700 1 -- 172.21.15.104:0/2300513342 wait complete. 2024-08-15T17:58:12.711 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new c87912dd-fe9f-4142-9c92-3ae704a8d4ec 2 2024-08-15T17:58:12.816 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.810+0000 7f7b77fff700 1 Processor -- start 2024-08-15T17:58:12.816 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.810+0000 7f7b77fff700 1 -- start start 2024-08-15T17:58:12.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7b78157140 0x7f7b780b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7b780b25a0 0x7f7b780698a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b780cba90 con 0x7f7b78156d70 2024-08-15T17:58:12.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b780b29b0 con 0x7f7b780b25a0 2024-08-15T17:58:12.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b780cae80 con 0x7f7b78157140 2024-08-15T17:58:12.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b767fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7b78157140 0x7f7b780b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b76ffd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f7b78156d70 0x7f7b780678c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:33230/0 (socket says 172.21.15.104:33230) 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b76ffd700 1 -- 172.21.15.104:0/3742102934 learned_addr learned my addr 172.21.15.104:0/3742102934 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b777fe700 1 --2- 172.21.15.104:0/3742102934 >> v2:172.21.15.38:3300/0 conn(0x7f7b780b25a0 0x7f7b780698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b767fc700 1 -- 172.21.15.104:0/3742102934 >> v2:172.21.15.38:3300/0 conn(0x7f7b780b25a0 msgr2=0x7f7b780698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b767fc700 1 --2- 172.21.15.104:0/3742102934 >> v2:172.21.15.38:3300/0 conn(0x7f7b780b25a0 0x7f7b780698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b75ffb700 1 -- 172.21.15.104:0/3742102934 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492179371 0 0) 0x7f7b780cba90 con 0x7f7b78156d70 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b767fc700 1 -- 172.21.15.104:0/3742102934 >> v1:172.21.15.104:6789/0 conn(0x7f7b78156d70 legacy=0x7f7b780678c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b767fc700 1 -- 172.21.15.104:0/3742102934 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b780cb160 con 0x7f7b78157140 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b767fc700 1 --2- 172.21.15.104:0/3742102934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b78157140 0x7f7b780b2060 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7b78064df0 tx=0x7f7b6c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11166eb0eb20e558 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b75ffb700 1 -- 172.21.15.104:0/3742102934 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c011630 con 0x7f7b78157140 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b75ffb700 1 -- 172.21.15.104:0/3742102934 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c011790 con 0x7f7b78157140 2024-08-15T17:58:12.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b75ffb700 1 -- 172.21.15.104:0/3742102934 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c011b30 con 0x7f7b78157140 2024-08-15T17:58:12.820 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 -- 172.21.15.104:0/3742102934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b78157140 msgr2=0x7f7b780b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.820 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 --2- 172.21.15.104:0/3742102934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b78157140 0x7f7b780b2060 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7b78064df0 tx=0x7f7b6c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.820 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 -- 172.21.15.104:0/3742102934 shutdown_connections 2024-08-15T17:58:12.820 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 --2- 172.21.15.104:0/3742102934 >> v2:172.21.15.38:3300/0 conn(0x7f7b780b25a0 0x7f7b780698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.820 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 --2- 172.21.15.104:0/3742102934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b78157140 0x7f7b780b2060 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.820 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 -- 172.21.15.104:0/3742102934 >> 172.21.15.104:0/3742102934 conn(0x7f7b7806e170 msgr2=0x7f7b7806e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:12.820 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 -- 172.21.15.104:0/3742102934 shutdown_connections 2024-08-15T17:58:12.821 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.814+0000 7f7b77fff700 1 -- 172.21.15.104:0/3742102934 wait complete. 2024-08-15T17:58:12.821 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b77fff700 1 Processor -- start 2024-08-15T17:58:12.821 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b77fff700 1 -- start start 2024-08-15T17:58:12.823 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b780b25a0 0x7f7b7814ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b78156d70 0x7f7b780ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b78157140 0x7f7b7816a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:12.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b77fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780aea50 con 0x7f7b780b25a0 2024-08-15T17:58:12.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b77fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7814fc50 con 0x7f7b78157140 2024-08-15T17:58:12.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b77fff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7814f040 con 0x7f7b78156d70 2024-08-15T17:58:12.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b767fc700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b78156d70 0x7f7b780ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b777fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b78157140 0x7f7b7816a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b767fc700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b78156d70 0x7f7b780ae510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.104:44510/0 (socket says 172.21.15.104:44510) 2024-08-15T17:58:12.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b767fc700 1 -- 172.21.15.104:0/1578136800 learned_addr learned my addr 172.21.15.104:0/1578136800 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b777fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b78157140 0x7f7b7816a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52944/0 (socket says 172.21.15.104:52944) 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b76ffd700 1 --2- 172.21.15.104:0/1578136800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b780b25a0 0x7f7b7814ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b777fe700 1 -- 172.21.15.104:0/1578136800 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b78156d70 msgr2=0x7f7b780ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b777fe700 1 --2- 172.21.15.104:0/1578136800 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b78156d70 0x7f7b780ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b777fe700 1 -- 172.21.15.104:0/1578136800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b780b25a0 msgr2=0x7f7b7814ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b777fe700 1 --2- 172.21.15.104:0/1578136800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b780b25a0 0x7f7b7814ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b777fe700 1 -- 172.21.15.104:0/1578136800 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c009420 con 0x7f7b78157140 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b777fe700 1 --2- 172.21.15.104:0/1578136800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b78157140 0x7f7b7816a120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009e50 tx=0x7f7b7000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b5effd700 1 -- 172.21.15.104:0/1578136800 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000caa0 con 0x7f7b78157140 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b5effd700 1 -- 172.21.15.104:0/1578136800 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7000f040 con 0x7f7b78157140 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b77fff700 1 -- 172.21.15.104:0/1578136800 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7814f320 con 0x7f7b78157140 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b5effd700 1 -- 172.21.15.104:0/1578136800 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000ce50 con 0x7f7b78157140 2024-08-15T17:58:12.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b77fff700 1 -- 172.21.15.104:0/1578136800 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7814f8e0 con 0x7f7b78157140 2024-08-15T17:58:12.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b5effd700 1 -- 172.21.15.104:0/1578136800 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f7b70014920 con 0x7f7b78157140 2024-08-15T17:58:12.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.818+0000 7f7b5effd700 1 -- 172.21.15.104:0/1578136800 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f7b7001d770 con 0x7f7b78157140 2024-08-15T17:58:12.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.822+0000 7f7b77fff700 1 -- 172.21.15.104:0/1578136800 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b780638f0 con 0x7f7b78157140 2024-08-15T17:58:12.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.822+0000 7f7b5effd700 1 -- 172.21.15.104:0/1578136800 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f7b7003b7a0 con 0x7f7b78157140 2024-08-15T17:58:12.888 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:12.882+0000 7fb9acbbc000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-15T17:58:12.904 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.898+0000 7f7b77fff700 1 -- 172.21.15.104:0/1578136800 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c87912dd-fe9f-4142-9c92-3ae704a8d4ec", "id": 2} v 0) v1 -- 0x7f7b7814fd90 con 0x7f7b78157140 2024-08-15T17:58:12.918 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.914+0000 7f7b5effd700 1 -- 172.21.15.104:0/1578136800 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c87912dd-fe9f-4142-9c92-3ae704a8d4ec", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7b7002a070 con 0x7f7b78157140 2024-08-15T17:58:12.918 INFO:teuthology.orchestra.run.smithi104.stdout:2 2024-08-15T17:58:12.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.914+0000 7f7b77fff700 1 -- 172.21.15.104:0/1578136800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b78157140 msgr2=0x7f7b7816a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:12.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.914+0000 7f7b77fff700 1 --2- 172.21.15.104:0/1578136800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b78157140 0x7f7b7816a120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009e50 tx=0x7f7b7000b040 comp rx=0 tx=0).stop 2024-08-15T17:58:12.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.914+0000 7f7b77fff700 1 -- 172.21.15.104:0/1578136800 shutdown_connections 2024-08-15T17:58:12.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.914+0000 7f7b77fff700 1 --2- 172.21.15.104:0/1578136800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b78157140 0x7f7b7816a120 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.914+0000 7f7b77fff700 1 --2- 172.21.15.104:0/1578136800 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b78156d70 0x7f7b780ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.914+0000 7f7b77fff700 1 --2- 172.21.15.104:0/1578136800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b780b25a0 0x7f7b7814ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:12.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.914+0000 7f7b77fff700 1 -- 172.21.15.104:0/1578136800 >> 172.21.15.104:0/1578136800 conn(0x7f7b7806e170 msgr2=0x7f7b78157670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:12.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.914+0000 7f7b77fff700 1 -- 172.21.15.104:0/1578136800 shutdown_connections 2024-08-15T17:58:12.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:12.914+0000 7f7b77fff700 1 -- 172.21.15.104:0/1578136800 wait complete. 2024-08-15T17:58:12.921 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:12.919+0000 7f5fa5693000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-15T17:58:12.933 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new 177bca56-b19b-4580-9bb4-483cabde4b91 3 2024-08-15T17:58:12.946 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:12.942+0000 7fb9acbbc000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-15T17:58:12.980 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:12.979+0000 7f5fa5693000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-15T17:58:13.037 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8d1ce6700 1 Processor -- start 2024-08-15T17:58:13.037 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8d1ce6700 1 -- start start 2024-08-15T17:58:13.039 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8d1ce6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc8cc1554c0 0x7fc8cc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.039 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8d1ce6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc069ee0 0x7fc8cc0686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.039 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8d1ce6700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8cc06b000 con 0x7fc8cc155890 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8d1ce6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8cc06a3f0 con 0x7fc8cc1554c0 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8d1ce6700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc06a6d0 con 0x7fc8cc069ee0 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8cb7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc8cc1554c0 0x7fc8cc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8cb7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc8cc1554c0 0x7fc8cc0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52948/0 (socket says 172.21.15.104:52948) 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8cb7fe700 1 -- 172.21.15.104:0/3313636885 learned_addr learned my addr 172.21.15.104:0/3313636885 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8caffd700 1 --2- 172.21.15.104:0/3313636885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc069ee0 0x7fc8cc0686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8cb7fe700 1 -- 172.21.15.104:0/3313636885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc069ee0 msgr2=0x7fc8cc0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8cb7fe700 1 --2- 172.21.15.104:0/3313636885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc069ee0 0x7fc8cc0686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8cb7fe700 1 -- 172.21.15.104:0/3313636885 >> v1:172.21.15.38:6789/0 conn(0x7fc8cc155890 legacy=0x7fc8cc159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8cb7fe700 1 -- 172.21.15.104:0/3313636885 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc06a9b0 con 0x7fc8cc1554c0 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8caffd700 1 --2- 172.21.15.104:0/3313636885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc069ee0 0x7fc8cc0686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8cb7fe700 1 --2- 172.21.15.104:0/3313636885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8cc1554c0 0x7fc8cc0699a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00bc50 tx=0x7fc8bc00b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7de98754e5fa062 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:13.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8ca7fc700 1 -- 172.21.15.104:0/3313636885 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc00dbf0 con 0x7fc8cc1554c0 2024-08-15T17:58:13.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8ca7fc700 1 -- 172.21.15.104:0/3313636885 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8bc00dd50 con 0x7fc8cc1554c0 2024-08-15T17:58:13.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8ca7fc700 1 -- 172.21.15.104:0/3313636885 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc0196b0 con 0x7fc8cc1554c0 2024-08-15T17:58:13.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3313636885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8cc1554c0 msgr2=0x7fc8cc0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.034+0000 7fc8d1ce6700 1 --2- 172.21.15.104:0/3313636885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8cc1554c0 0x7fc8cc0699a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00bc50 tx=0x7fc8bc00b4f0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3313636885 shutdown_connections 2024-08-15T17:58:13.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 --2- 172.21.15.104:0/3313636885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc069ee0 0x7fc8cc0686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 --2- 172.21.15.104:0/3313636885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8cc1554c0 0x7fc8cc0699a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3313636885 >> 172.21.15.104:0/3313636885 conn(0x7fc8cc0c9bf0 msgr2=0x7fc8cc06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:13.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3313636885 shutdown_connections 2024-08-15T17:58:13.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3313636885 wait complete. 2024-08-15T17:58:13.042 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 Processor -- start 2024-08-15T17:58:13.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 -- start start 2024-08-15T17:58:13.045 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8cc069ee0 0x7fc8cc156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.045 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc1554c0 0x7fc8cc16a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.045 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc155890 0x7fc8cc158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.045 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc0c7a00 con 0x7fc8cc1554c0 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc16bf30 con 0x7fc8cc069ee0 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc16ac30 con 0x7fc8cc155890 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8cbfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc155890 0x7fc8cc158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8cb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8cc069ee0 0x7fc8cc156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8cbfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc155890 0x7fc8cc158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.104:44538/0 (socket says 172.21.15.104:44538) 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8cbfff700 1 -- 172.21.15.104:0/3609583948 learned_addr learned my addr 172.21.15.104:0/3609583948 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8cb7fe700 1 --2- 172.21.15.104:0/3609583948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8cc069ee0 0x7fc8cc156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52964/0 (socket says 172.21.15.104:52964) 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8caffd700 1 --2- 172.21.15.104:0/3609583948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc1554c0 0x7fc8cc16a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8cbfff700 1 -- 172.21.15.104:0/3609583948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8cc069ee0 msgr2=0x7fc8cc156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8cbfff700 1 --2- 172.21.15.104:0/3609583948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8cc069ee0 0x7fc8cc156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8cbfff700 1 -- 172.21.15.104:0/3609583948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc1554c0 msgr2=0x7fc8cc16a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8cbfff700 1 --2- 172.21.15.104:0/3609583948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc1554c0 0x7fc8cc16a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8cbfff700 1 -- 172.21.15.104:0/3609583948 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c0009420 con 0x7fc8cc155890 2024-08-15T17:58:13.047 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8cbfff700 1 --2- 172.21.15.104:0/3609583948 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc155890 0x7fc8cc158ae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400c910 tx=0x7fc8c4001710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:13.047 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8b37fe700 1 -- 172.21.15.104:0/3609583948 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4012630 con 0x7fc8cc155890 2024-08-15T17:58:13.047 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8b37fe700 1 -- 172.21.15.104:0/3609583948 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c40127b0 con 0x7fc8cc155890 2024-08-15T17:58:13.047 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3609583948 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8bc00b790 con 0x7fc8cc155890 2024-08-15T17:58:13.047 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8b37fe700 1 -- 172.21.15.104:0/3609583948 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4012b70 con 0x7fc8cc155890 2024-08-15T17:58:13.047 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3609583948 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc16b600 con 0x7fc8cc155890 2024-08-15T17:58:13.047 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8b37fe700 1 -- 172.21.15.104:0/3609583948 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fc8c4003030 con 0x7fc8cc155890 2024-08-15T17:58:13.047 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.038+0000 7fc8b37fe700 1 -- 172.21.15.104:0/3609583948 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fc8c401a3a0 con 0x7fc8cc155890 2024-08-15T17:58:13.047 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.042+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3609583948 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8cc06d420 con 0x7fc8cc155890 2024-08-15T17:58:13.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.046+0000 7fc8b37fe700 1 -- 172.21.15.104:0/3609583948 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fc8cc06d420 con 0x7fc8cc155890 2024-08-15T17:58:13.068 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:13.062+0000 7fb9acbbc000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-15T17:58:13.099 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:13.099+0000 7f5fa5693000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-15T17:58:13.126 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.122+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3609583948 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd new", "uuid": "177bca56-b19b-4580-9bb4-483cabde4b91", "id": 3} v 0) v1 -- 0x7fc8cc0645c0 con 0x7fc8cc155890 2024-08-15T17:58:13.132 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.130+0000 7fc8b37fe700 1 -- 172.21.15.104:0/3609583948 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "177bca56-b19b-4580-9bb4-483cabde4b91", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc8c401b4c0 con 0x7fc8cc155890 2024-08-15T17:58:13.133 INFO:teuthology.orchestra.run.smithi104.stdout:3 2024-08-15T17:58:13.134 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.130+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3609583948 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc155890 msgr2=0x7fc8cc158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.134 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.130+0000 7fc8d1ce6700 1 --2- 172.21.15.104:0/3609583948 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc155890 0x7fc8cc158ae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400c910 tx=0x7fc8c4001710 comp rx=0 tx=0).stop 2024-08-15T17:58:13.134 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.130+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3609583948 shutdown_connections 2024-08-15T17:58:13.134 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.130+0000 7fc8d1ce6700 1 --2- 172.21.15.104:0/3609583948 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8cc155890 0x7fc8cc158ae0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.134 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.130+0000 7fc8d1ce6700 1 --2- 172.21.15.104:0/3609583948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc1554c0 0x7fc8cc16a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.134 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.130+0000 7fc8d1ce6700 1 --2- 172.21.15.104:0/3609583948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8cc069ee0 0x7fc8cc156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.130+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3609583948 >> 172.21.15.104:0/3609583948 conn(0x7fc8cc0c9bf0 msgr2=0x7fc8cc0af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:13.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.130+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3609583948 shutdown_connections 2024-08-15T17:58:13.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.130+0000 7fc8d1ce6700 1 -- 172.21.15.104:0/3609583948 wait complete. 2024-08-15T17:58:13.145 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new 959df71d-f02f-4b4f-a79a-dd9194728ea6 4 2024-08-15T17:58:13.247 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.242+0000 7f90b759e700 1 Processor -- start 2024-08-15T17:58:13.247 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.242+0000 7f90b759e700 1 -- start start 2024-08-15T17:58:13.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.242+0000 7f90b759e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f90b80ae9e0 0x7f90b80b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.242+0000 7f90b759e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b8068620 con 0x7f90b8142980 2024-08-15T17:58:13.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.242+0000 7f90b759e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b8068be0 con 0x7f90b80b12b0 2024-08-15T17:58:13.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.242+0000 7f90b759e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b80b16f0 con 0x7f90b80ae9e0 2024-08-15T17:58:13.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.242+0000 7f90b6d9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f90b80b12b0 0x7f90b8069070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:33240/0 (socket says 172.21.15.104:33240) 2024-08-15T17:58:13.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.242+0000 7f90b6d9d700 1 -- 172.21.15.104:0/3049028816 learned_addr learned my addr 172.21.15.104:0/3049028816 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-08-15T17:58:13.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.242+0000 7f90b5d9b700 1 --2- 172.21.15.104:0/3049028816 >> v2:172.21.15.38:3300/0 conn(0x7f90b80ae9e0 0x7f90b80b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.242+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1867281247 0 0) 0x7f90b8068be0 con 0x7f90b80b12b0 2024-08-15T17:58:13.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.242+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90a0003570 con 0x7f90b80b12b0 2024-08-15T17:58:13.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 <== mon.1 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2779948730 0 0) 0x7f90a0003570 con 0x7f90b80b12b0 2024-08-15T17:58:13.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 >> v2:172.21.15.38:3300/0 conn(0x7f90b80ae9e0 msgr2=0x7f90b80b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:13.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 --2- 172.21.15.104:0/3049028816 >> v2:172.21.15.38:3300/0 conn(0x7f90b80ae9e0 0x7f90b80b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 >> v1:172.21.15.38:6789/0 conn(0x7f90b8142980 legacy=0x7f90b815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b815d7e0 con 0x7f90b80b12b0 2024-08-15T17:58:13.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 <== mon.1 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f90b00017f0 con 0x7f90b80b12b0 2024-08-15T17:58:13.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 >> v1:172.21.15.104:6789/0 conn(0x7f90b80b12b0 legacy=0x7f90b8069070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90a0003a60 0x7f90a0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 0x7f90a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a00051c0 0x7f90a000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b80b16f0 con 0x7f90a0003a60 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a000c160 con 0x7f90a0004810 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a000c320 con 0x7f90a00051c0 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b6d9d700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a00051c0 0x7f90a000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b5d9b700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 0x7f90a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b659c700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90a0003a60 0x7f90a0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b5d9b700 1 -- 172.21.15.104:0/3049028816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a00051c0 msgr2=0x7f90a000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b5d9b700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a00051c0 0x7f90a000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b5d9b700 1 -- 172.21.15.104:0/3049028816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90a0003a60 msgr2=0x7f90a0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b5d9b700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90a0003a60 0x7f90a0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b5d9b700 1 -- 172.21.15.104:0/3049028816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0003570 con 0x7f90a0004810 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b6d9d700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a00051c0 0x7f90a000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:13.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b6d9d700 1 -- 172.21.15.104:0/3049028816 reap_dead start 2024-08-15T17:58:13.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b5d9b700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 0x7f90a0004c80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009c00 tx=0x7f90ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa8546b51db92413 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:13.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00e710 con 0x7f90a0004810 2024-08-15T17:58:13.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ac00e870 con 0x7f90a0004810 2024-08-15T17:58:13.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b559a700 1 -- 172.21.15.104:0/3049028816 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00ec30 con 0x7f90a0004810 2024-08-15T17:58:13.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b759e700 1 -- 172.21.15.104:0/3049028816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 msgr2=0x7f90a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b759e700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 0x7f90a0004c80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009c00 tx=0x7f90ac0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b759e700 1 -- 172.21.15.104:0/3049028816 shutdown_connections 2024-08-15T17:58:13.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b759e700 1 --2- 172.21.15.104:0/3049028816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 0x7f90a0004c80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b759e700 1 -- 172.21.15.104:0/3049028816 >> 172.21.15.104:0/3049028816 conn(0x7f90b806cbe0 msgr2=0x7f90b806cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:13.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b759e700 1 -- 172.21.15.104:0/3049028816 shutdown_connections 2024-08-15T17:58:13.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.246+0000 7f90b759e700 1 -- 172.21.15.104:0/3049028816 wait complete. 2024-08-15T17:58:13.256 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b759e700 1 Processor -- start 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b759e700 1 -- start start 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 0x7f90b813faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b759e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90b80cb910 0x7f90b8141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b759e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90b80cbd80 0x7f90b8141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b759e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b80ca3f0 con 0x7f90b80cb910 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b80c7230 con 0x7f90a0004810 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b759e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b80c7510 con 0x7f90b80cbd80 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 0x7f90b813faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 0x7f90b813faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52978/0 (socket says 172.21.15.104:52978) 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b659c700 1 -- 172.21.15.104:0/1875181419 learned_addr learned my addr 172.21.15.104:0/1875181419 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b6d9d700 1 --2- 172.21.15.104:0/1875181419 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90b80cbd80 0x7f90b8141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b5d9b700 1 --2- 172.21.15.104:0/1875181419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90b80cb910 0x7f90b8141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b659c700 1 -- 172.21.15.104:0/1875181419 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90b80cbd80 msgr2=0x7f90b8141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b659c700 1 --2- 172.21.15.104:0/1875181419 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90b80cbd80 0x7f90b8141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b659c700 1 -- 172.21.15.104:0/1875181419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90b80cb910 msgr2=0x7f90b8141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b659c700 1 --2- 172.21.15.104:0/1875181419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90b80cb910 0x7f90b8141610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b659c700 1 -- 172.21.15.104:0/1875181419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b80c77f0 con 0x7f90a0004810 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b659c700 1 --2- 172.21.15.104:0/1875181419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 0x7f90b813faf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90a8002920 tx=0x7f90a8007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90a67fc700 1 -- 172.21.15.104:0/1875181419 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a8003410 con 0x7f90a0004810 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90a67fc700 1 -- 172.21.15.104:0/1875181419 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a800f8d0 con 0x7f90a0004810 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b759e700 1 -- 172.21.15.104:0/1875181419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b80c7ad0 con 0x7f90a0004810 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90a67fc700 1 -- 172.21.15.104:0/1875181419 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a800fc70 con 0x7f90a0004810 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90b759e700 1 -- 172.21.15.104:0/1875181419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b80c7ff0 con 0x7f90a0004810 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90a67fc700 1 -- 172.21.15.104:0/1875181419 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f90a8013890 con 0x7f90a0004810 2024-08-15T17:58:13.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.250+0000 7f90a67fc700 1 -- 172.21.15.104:0/1875181419 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f90a8013b40 con 0x7f90a0004810 2024-08-15T17:58:13.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.254+0000 7f90b759e700 1 -- 172.21.15.104:0/1875181419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b80638f0 con 0x7f90a0004810 2024-08-15T17:58:13.260 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.254+0000 7f90a67fc700 1 -- 172.21.15.104:0/1875181419 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f90a8017020 con 0x7f90a0004810 2024-08-15T17:58:13.303 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:13.298+0000 7fb9acbbc000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-15T17:58:13.335 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.330+0000 7f90b759e700 1 -- 172.21.15.104:0/1875181419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "959df71d-f02f-4b4f-a79a-dd9194728ea6", "id": 4} v 0) v1 -- 0x7f90b80cd980 con 0x7f90a0004810 2024-08-15T17:58:13.345 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.342+0000 7f90a67fc700 1 -- 172.21.15.104:0/1875181419 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "959df71d-f02f-4b4f-a79a-dd9194728ea6", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f90a802a020 con 0x7f90a0004810 2024-08-15T17:58:13.345 INFO:teuthology.orchestra.run.smithi104.stdout:4 2024-08-15T17:58:13.346 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:13.343+0000 7f5fa5693000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-15T17:58:13.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.342+0000 7f90b759e700 1 -- 172.21.15.104:0/1875181419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 msgr2=0x7f90b813faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.342+0000 7f90b759e700 1 --2- 172.21.15.104:0/1875181419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 0x7f90b813faf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90a8002920 tx=0x7f90a8007d30 comp rx=0 tx=0).stop 2024-08-15T17:58:13.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.342+0000 7f90b759e700 1 -- 172.21.15.104:0/1875181419 shutdown_connections 2024-08-15T17:58:13.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.342+0000 7f90b759e700 1 --2- 172.21.15.104:0/1875181419 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90b80cbd80 0x7f90b8141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.342+0000 7f90b759e700 1 --2- 172.21.15.104:0/1875181419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90b80cb910 0x7f90b8141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.342+0000 7f90b759e700 1 --2- 172.21.15.104:0/1875181419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a0004810 0x7f90b813faf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.342+0000 7f90b759e700 1 -- 172.21.15.104:0/1875181419 >> 172.21.15.104:0/1875181419 conn(0x7f90b806cbe0 msgr2=0x7f90b80afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:13.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.342+0000 7f90b759e700 1 -- 172.21.15.104:0/1875181419 shutdown_connections 2024-08-15T17:58:13.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.342+0000 7f90b759e700 1 -- 172.21.15.104:0/1875181419 wait complete. 2024-08-15T17:58:13.351 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:13.346+0000 7fb9acbbc000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-15T17:58:13.358 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new 023ef8e6-1845-4938-afaa-eeb0ea8e135f 5 2024-08-15T17:58:13.400 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:13.399+0000 7f5fa5693000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-15T17:58:13.451 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460ff6b700 1 Processor -- start 2024-08-15T17:58:13.451 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460ff6b700 1 -- start start 2024-08-15T17:58:13.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460ff6b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 0x7f46080cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460ff6b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46080cd840 con 0x7f46080cd3d0 2024-08-15T17:58:13.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460ff6b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46080cca00 con 0x7f46080cf1f0 2024-08-15T17:58:13.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460ff6b700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46080b2e80 con 0x7f460815cb60 2024-08-15T17:58:13.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460dd07700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 0x7f46080cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460dd07700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 0x7f46080cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.104:44564/0 (socket says 172.21.15.104:44564) 2024-08-15T17:58:13.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460dd07700 1 -- 172.21.15.104:0/3942477107 learned_addr learned my addr 172.21.15.104:0/3942477107 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:13.453 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460cd05700 1 -- 172.21.15.104:0/3942477107 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661568383 0 0) 0x7f46080cca00 con 0x7f46080cf1f0 2024-08-15T17:58:13.453 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460dd07700 1 -- 172.21.15.104:0/3942477107 >> v1:172.21.15.104:6789/0 conn(0x7f46080cf1f0 legacy=0x7f46081631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.453 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460dd07700 1 -- 172.21.15.104:0/3942477107 >> v1:172.21.15.38:6789/0 conn(0x7f46080cd3d0 legacy=0x7f46080ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.453 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.446+0000 7f460dd07700 1 -- 172.21.15.104:0/3942477107 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460815cf30 con 0x7f460815cb60 2024-08-15T17:58:13.454 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460dd07700 1 --2- 172.21.15.104:0/3942477107 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 0x7f46080cce90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f45f80099e0 tx=0x7f45f8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2945a8c29502fc server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:13.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460cd05700 1 -- 172.21.15.104:0/3942477107 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800b860 con 0x7f460815cb60 2024-08-15T17:58:13.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460cd05700 1 -- 172.21.15.104:0/3942477107 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f800b9c0 con 0x7f460815cb60 2024-08-15T17:58:13.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460cd05700 1 -- 172.21.15.104:0/3942477107 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800bd60 con 0x7f460815cb60 2024-08-15T17:58:13.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460ff6b700 1 -- 172.21.15.104:0/3942477107 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 msgr2=0x7f46080cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460ff6b700 1 --2- 172.21.15.104:0/3942477107 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 0x7f46080cce90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f45f80099e0 tx=0x7f45f8009250 comp rx=0 tx=0).stop 2024-08-15T17:58:13.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460ff6b700 1 -- 172.21.15.104:0/3942477107 shutdown_connections 2024-08-15T17:58:13.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460ff6b700 1 --2- 172.21.15.104:0/3942477107 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 0x7f46080cce90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460ff6b700 1 -- 172.21.15.104:0/3942477107 >> 172.21.15.104:0/3942477107 conn(0x7f46080996d0 msgr2=0x7f46080675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:13.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460ff6b700 1 -- 172.21.15.104:0/3942477107 shutdown_connections 2024-08-15T17:58:13.456 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460ff6b700 1 -- 172.21.15.104:0/3942477107 wait complete. 2024-08-15T17:58:13.456 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460ff6b700 1 Processor -- start 2024-08-15T17:58:13.456 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.450+0000 7f460ff6b700 1 -- start start 2024-08-15T17:58:13.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460ff6b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46080cd3d0 0x7f4608153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460ff6b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080cf1f0 0x7f460816a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460ff6b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 0x7f46081588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460ff6b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080ca740 con 0x7f46080cf1f0 2024-08-15T17:58:13.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460ff6b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080c7fa0 con 0x7f46080cd3d0 2024-08-15T17:58:13.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460ff6b700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4608153560 con 0x7f460815cb60 2024-08-15T17:58:13.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460dd07700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46080cd3d0 0x7f4608153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460dd07700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46080cd3d0 0x7f4608153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52988/0 (socket says 172.21.15.104:52988) 2024-08-15T17:58:13.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460dd07700 1 -- 172.21.15.104:0/1366733576 learned_addr learned my addr 172.21.15.104:0/1366733576 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:13.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460e508700 1 --2- 172.21.15.104:0/1366733576 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 0x7f46081588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460d506700 1 --2- 172.21.15.104:0/1366733576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080cf1f0 0x7f460816a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460dd07700 1 -- 172.21.15.104:0/1366733576 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 msgr2=0x7f46081588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460dd07700 1 --2- 172.21.15.104:0/1366733576 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 0x7f46081588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460dd07700 1 -- 172.21.15.104:0/1366733576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080cf1f0 msgr2=0x7f460816a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460dd07700 1 --2- 172.21.15.104:0/1366733576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080cf1f0 0x7f460816a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460dd07700 1 -- 172.21.15.104:0/1366733576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f80094f0 con 0x7f46080cd3d0 2024-08-15T17:58:13.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460d506700 1 --2- 172.21.15.104:0/1366733576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080cf1f0 0x7f460816a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:13.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460dd07700 1 --2- 172.21.15.104:0/1366733576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46080cd3d0 0x7f4608153020 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c010 tx=0x7f45f8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:13.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f4605ffb700 1 -- 172.21.15.104:0/1366733576 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f801f070 con 0x7f46080cd3d0 2024-08-15T17:58:13.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f4605ffb700 1 -- 172.21.15.104:0/1366733576 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f800bd40 con 0x7f46080cd3d0 2024-08-15T17:58:13.461 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f4605ffb700 1 -- 172.21.15.104:0/1366733576 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f801a630 con 0x7f46080cd3d0 2024-08-15T17:58:13.461 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460ff6b700 1 -- 172.21.15.104:0/1366733576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46080c7670 con 0x7f46080cd3d0 2024-08-15T17:58:13.461 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460ff6b700 1 -- 172.21.15.104:0/1366733576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46080c7c30 con 0x7f46080cd3d0 2024-08-15T17:58:13.461 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f4605ffb700 1 -- 172.21.15.104:0/1366733576 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f45f801a9a0 con 0x7f46080cd3d0 2024-08-15T17:58:13.461 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f4605ffb700 1 -- 172.21.15.104:0/1366733576 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f45f80239e0 con 0x7f46080cd3d0 2024-08-15T17:58:13.461 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.454+0000 7f460ff6b700 1 -- 172.21.15.104:0/1366733576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46080645c0 con 0x7f46080cd3d0 2024-08-15T17:58:13.463 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.458+0000 7f4605ffb700 1 -- 172.21.15.104:0/1366733576 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f45f8031060 con 0x7f46080cd3d0 2024-08-15T17:58:13.527 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:13.522+0000 7fb9acbbc000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-15T17:58:13.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.538+0000 7f460ff6b700 1 -- 172.21.15.104:0/1366733576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "023ef8e6-1845-4938-afaa-eeb0ea8e135f", "id": 5} v 0) v1 -- 0x7f4608071590 con 0x7f46080cd3d0 2024-08-15T17:58:13.555 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.550+0000 7f4605ffb700 1 -- 172.21.15.104:0/1366733576 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "023ef8e6-1845-4938-afaa-eeb0ea8e135f", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f45f8028070 con 0x7f46080cd3d0 2024-08-15T17:58:13.555 INFO:teuthology.orchestra.run.smithi104.stdout:5 2024-08-15T17:58:13.556 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.550+0000 7f460ff6b700 1 -- 172.21.15.104:0/1366733576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46080cd3d0 msgr2=0x7f4608153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.556 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.550+0000 7f460ff6b700 1 --2- 172.21.15.104:0/1366733576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46080cd3d0 0x7f4608153020 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c010 tx=0x7f45f8017000 comp rx=0 tx=0).stop 2024-08-15T17:58:13.556 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.550+0000 7f460ff6b700 1 -- 172.21.15.104:0/1366733576 shutdown_connections 2024-08-15T17:58:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.550+0000 7f460ff6b700 1 --2- 172.21.15.104:0/1366733576 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f460815cb60 0x7f46081588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.550+0000 7f460ff6b700 1 --2- 172.21.15.104:0/1366733576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080cf1f0 0x7f460816a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.550+0000 7f460ff6b700 1 --2- 172.21.15.104:0/1366733576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46080cd3d0 0x7f4608153020 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.550+0000 7f460ff6b700 1 -- 172.21.15.104:0/1366733576 >> 172.21.15.104:0/1366733576 conn(0x7f46080996d0 msgr2=0x7f46080c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.550+0000 7f460ff6b700 1 -- 172.21.15.104:0/1366733576 shutdown_connections 2024-08-15T17:58:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.550+0000 7f460ff6b700 1 -- 172.21.15.104:0/1366733576 wait complete. 2024-08-15T17:58:13.569 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new 750271d8-321b-47bf-ad2e-2b5e6de7d983 6 2024-08-15T17:58:13.589 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:13.586+0000 7fb9acbbc000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-15T17:58:13.597 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:13.595+0000 7f5fa5693000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-15T17:58:13.652 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:13.646+0000 7fb9acbbc000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-15T17:58:13.664 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:13.663+0000 7f5fa5693000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-15T17:58:13.667 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.662+0000 7f6a9de3b700 1 Processor -- start 2024-08-15T17:58:13.667 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.662+0000 7f6a9de3b700 1 -- start start 2024-08-15T17:58:13.670 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6a98159fb0 0x7f6a9815a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.670 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6a98155820 0x7f6a98155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.670 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a980ac7f0 con 0x7f6a98159be0 2024-08-15T17:58:13.670 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a980c8580 con 0x7f6a98159fb0 2024-08-15T17:58:13.670 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a980c8860 con 0x7f6a98155820 2024-08-15T17:58:13.670 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9d63a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6a98155820 0x7f6a98155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.671 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9d63a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6a98155820 0x7f6a98155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52994/0 (socket says 172.21.15.104:52994) 2024-08-15T17:58:13.671 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9d63a700 1 -- 172.21.15.104:0/3466924155 learned_addr learned my addr 172.21.15.104:0/3466924155 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:13.671 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a97fff700 1 --2- 172.21.15.104:0/3466924155 >> v2:172.21.15.38:3300/0 conn(0x7f6a98159fb0 0x7f6a9815a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.671 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9d63a700 1 -- 172.21.15.104:0/3466924155 >> v2:172.21.15.38:3300/0 conn(0x7f6a98159fb0 msgr2=0x7f6a9815a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:13.671 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9d63a700 1 --2- 172.21.15.104:0/3466924155 >> v2:172.21.15.38:3300/0 conn(0x7f6a98159fb0 0x7f6a9815a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.671 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9d63a700 1 -- 172.21.15.104:0/3466924155 >> v1:172.21.15.38:6789/0 conn(0x7f6a98159be0 legacy=0x7f6a980c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.671 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9d63a700 1 -- 172.21.15.104:0/3466924155 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a980c8b40 con 0x7f6a98155820 2024-08-15T17:58:13.671 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9d63a700 1 --2- 172.21.15.104:0/3466924155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a98155820 0x7f6a98155c30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a980c7380 tx=0x7f6a90009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14d0ed6aede2b8ef server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:13.672 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a977fe700 1 -- 172.21.15.104:0/3466924155 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000e070 con 0x7f6a98155820 2024-08-15T17:58:13.672 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a977fe700 1 -- 172.21.15.104:0/3466924155 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9000ba20 con 0x7f6a98155820 2024-08-15T17:58:13.672 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a977fe700 1 -- 172.21.15.104:0/3466924155 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a900092e0 con 0x7f6a98155820 2024-08-15T17:58:13.672 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 -- 172.21.15.104:0/3466924155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a98155820 msgr2=0x7f6a98155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.672 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 --2- 172.21.15.104:0/3466924155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a98155820 0x7f6a98155c30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a980c7380 tx=0x7f6a90009840 comp rx=0 tx=0).stop 2024-08-15T17:58:13.673 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 -- 172.21.15.104:0/3466924155 shutdown_connections 2024-08-15T17:58:13.673 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 --2- 172.21.15.104:0/3466924155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a98155820 0x7f6a98155c30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.673 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 --2- 172.21.15.104:0/3466924155 >> v2:172.21.15.38:3300/0 conn(0x7f6a98159fb0 0x7f6a9815a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.673 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 -- 172.21.15.104:0/3466924155 >> 172.21.15.104:0/3466924155 conn(0x7f6a980c9830 msgr2=0x7f6a980c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:13.673 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 -- 172.21.15.104:0/3466924155 shutdown_connections 2024-08-15T17:58:13.673 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.666+0000 7f6a9de3b700 1 -- 172.21.15.104:0/3466924155 wait complete. 2024-08-15T17:58:13.674 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9de3b700 1 Processor -- start 2024-08-15T17:58:13.674 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9de3b700 1 -- start start 2024-08-15T17:58:13.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9de3b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6a98155820 0x7f6a981468f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9de3b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a98159be0 0x7f6a98158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9de3b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a98159fb0 0x7f6a980c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9de3b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980c2d50 con 0x7f6a98159be0 2024-08-15T17:58:13.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9de3b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a98146e30 con 0x7f6a98159fb0 2024-08-15T17:58:13.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9de3b700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a98165cf0 con 0x7f6a98155820 2024-08-15T17:58:13.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9ce39700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6a98155820 0x7f6a981468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9d63a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a98159fb0 0x7f6a980c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9ce39700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6a98155820 0x7f6a981468f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.104:44582/0 (socket says 172.21.15.104:44582) 2024-08-15T17:58:13.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9ce39700 1 -- 172.21.15.104:0/1879651767 learned_addr learned my addr 172.21.15.104:0/1879651767 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a97fff700 1 --2- 172.21.15.104:0/1879651767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a98159be0 0x7f6a98158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9d63a700 1 -- 172.21.15.104:0/1879651767 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6a98155820 msgr2=0x7f6a981468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9d63a700 1 --2- 172.21.15.104:0/1879651767 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6a98155820 0x7f6a981468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9d63a700 1 -- 172.21.15.104:0/1879651767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a98159be0 msgr2=0x7f6a98158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9d63a700 1 --2- 172.21.15.104:0/1879651767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a98159be0 0x7f6a98158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9d63a700 1 -- 172.21.15.104:0/1879651767 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a90009ae0 con 0x7f6a98159fb0 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.670+0000 7f6a9ce39700 1 --2- 172.21.15.104:0/1879651767 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6a98155820 0x7f6a981468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.674+0000 7f6a9d63a700 1 --2- 172.21.15.104:0/1879651767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a98159fb0 0x7f6a980c9c00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a90015000 tx=0x7f6a90018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.674+0000 7f6a94ff9700 1 -- 172.21.15.104:0/1879651767 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90008a80 con 0x7f6a98159fb0 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.674+0000 7f6a94ff9700 1 -- 172.21.15.104:0/1879651767 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a90008c00 con 0x7f6a98159fb0 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.674+0000 7f6a9de3b700 1 -- 172.21.15.104:0/1879651767 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a98158fb0 con 0x7f6a98159fb0 2024-08-15T17:58:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.674+0000 7f6a94ff9700 1 -- 172.21.15.104:0/1879651767 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90026460 con 0x7f6a98159fb0 2024-08-15T17:58:13.680 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.674+0000 7f6a9de3b700 1 -- 172.21.15.104:0/1879651767 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a98157790 con 0x7f6a98159fb0 2024-08-15T17:58:13.680 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.674+0000 7f6a94ff9700 1 -- 172.21.15.104:0/1879651767 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f6a90003040 con 0x7f6a98159fb0 2024-08-15T17:58:13.680 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.674+0000 7f6a94ff9700 1 -- 172.21.15.104:0/1879651767 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f6a90026a60 con 0x7f6a98159fb0 2024-08-15T17:58:13.680 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.674+0000 7f6a9de3b700 1 -- 172.21.15.104:0/1879651767 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a98165ee0 con 0x7f6a98159fb0 2024-08-15T17:58:13.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.678+0000 7f6a94ff9700 1 -- 172.21.15.104:0/1879651767 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f6a90021070 con 0x7f6a98159fb0 2024-08-15T17:58:13.733 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:13.731+0000 7f5fa5693000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-15T17:58:13.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.750+0000 7f6a9de3b700 1 -- 172.21.15.104:0/1879651767 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "750271d8-321b-47bf-ad2e-2b5e6de7d983", "id": 6} v 0) v1 -- 0x7f6a980638f0 con 0x7f6a98159fb0 2024-08-15T17:58:13.764 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.758+0000 7f6a94ff9700 1 -- 172.21.15.104:0/1879651767 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "750271d8-321b-47bf-ad2e-2b5e6de7d983", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6a90026ce0 con 0x7f6a98159fb0 2024-08-15T17:58:13.765 INFO:teuthology.orchestra.run.smithi104.stdout:6 2024-08-15T17:58:13.766 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.762+0000 7f6a9de3b700 1 -- 172.21.15.104:0/1879651767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a98159fb0 msgr2=0x7f6a980c9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.766 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.762+0000 7f6a9de3b700 1 --2- 172.21.15.104:0/1879651767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a98159fb0 0x7f6a980c9c00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a90015000 tx=0x7f6a90018000 comp rx=0 tx=0).stop 2024-08-15T17:58:13.766 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.762+0000 7f6a9de3b700 1 -- 172.21.15.104:0/1879651767 shutdown_connections 2024-08-15T17:58:13.766 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.762+0000 7f6a9de3b700 1 --2- 172.21.15.104:0/1879651767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a98159fb0 0x7f6a980c9c00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.766 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.762+0000 7f6a9de3b700 1 --2- 172.21.15.104:0/1879651767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a98159be0 0x7f6a98158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.766 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.762+0000 7f6a9de3b700 1 --2- 172.21.15.104:0/1879651767 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6a98155820 0x7f6a981468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.767 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.762+0000 7f6a9de3b700 1 -- 172.21.15.104:0/1879651767 >> 172.21.15.104:0/1879651767 conn(0x7f6a980c9830 msgr2=0x7f6a98154700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:13.767 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.762+0000 7f6a9de3b700 1 -- 172.21.15.104:0/1879651767 shutdown_connections 2024-08-15T17:58:13.767 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.762+0000 7f6a9de3b700 1 -- 172.21.15.104:0/1879651767 wait complete. 2024-08-15T17:58:13.779 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new ea9c9c4a-9662-4a09-b81f-260ccc5c27d4 7 2024-08-15T17:58:13.880 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.874+0000 7f15bc36d700 1 Processor -- start 2024-08-15T17:58:13.880 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.874+0000 7f15bc36d700 1 -- start start 2024-08-15T17:58:13.884 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15bc36d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f15b415a500 0x7f15b40ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.884 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15bc36d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b406ed10 con 0x7f15b4068db0 2024-08-15T17:58:13.884 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15bc36d700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b40b1dc0 con 0x7f15b415a8d0 2024-08-15T17:58:13.884 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15bc36d700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b406ea30 con 0x7f15b415a500 2024-08-15T17:58:13.884 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9908700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f15b415a8d0 0x7f15b40686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:33252/0 (socket says 172.21.15.104:33252) 2024-08-15T17:58:13.884 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9908700 1 -- 172.21.15.104:0/616480278 learned_addr learned my addr 172.21.15.104:0/616480278 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-08-15T17:58:13.884 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15ba109700 1 --2- 172.21.15.104:0/616480278 >> v2:172.21.15.38:3300/0 conn(0x7f15b415a500 0x7f15b40ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.884 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269954764 0 0) 0x7f15b40b1dc0 con 0x7f15b415a8d0 2024-08-15T17:58:13.884 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15a0003570 con 0x7f15b415a8d0 2024-08-15T17:58:13.884 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 <== mon.1 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2991342380 0 0) 0x7f15a0003570 con 0x7f15b415a8d0 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 >> v2:172.21.15.38:3300/0 conn(0x7f15b415a500 msgr2=0x7f15b40ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 --2- 172.21.15.104:0/616480278 >> v2:172.21.15.38:3300/0 conn(0x7f15b415a500 0x7f15b40ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 >> v1:172.21.15.38:6789/0 conn(0x7f15b4068db0 legacy=0x7f15b40b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b40b1770 con 0x7f15b415a8d0 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 <== mon.1 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f15a4001470 con 0x7f15b415a8d0 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 >> v1:172.21.15.104:6789/0 conn(0x7f15b415a8d0 legacy=0x7f15b40686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 --2- 172.21.15.104:0/616480278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15a0003a60 0x7f15a0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 --2- 172.21.15.104:0/616480278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0004810 0x7f15a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 --2- 172.21.15.104:0/616480278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15a00051c0 0x7f15a000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b406ea30 con 0x7f15a00051c0 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a000c0d0 con 0x7f15a0004810 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a000c290 con 0x7f15a0003a60 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15ba109700 1 --2- 172.21.15.104:0/616480278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15a0003a60 0x7f15a0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9908700 1 --2- 172.21.15.104:0/616480278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0004810 0x7f15a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15ba90a700 1 --2- 172.21.15.104:0/616480278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15a00051c0 0x7f15a000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15ba109700 1 -- 172.21.15.104:0/616480278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0004810 msgr2=0x7f15a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:13.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15ba109700 1 --2- 172.21.15.104:0/616480278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0004810 0x7f15a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15ba109700 1 -- 172.21.15.104:0/616480278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15a00051c0 msgr2=0x7f15a000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:13.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15ba109700 1 --2- 172.21.15.104:0/616480278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15a00051c0 0x7f15a000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15ba90a700 1 -- 172.21.15.104:0/616480278 reap_dead start 2024-08-15T17:58:13.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15ba109700 1 -- 172.21.15.104:0/616480278 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0003570 con 0x7f15a0003a60 2024-08-15T17:58:13.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15ba109700 1 --2- 172.21.15.104:0/616480278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15a0003a60 0x7f15a0003e70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009c00 tx=0x7f15b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=537268d02ab61851 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:13.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b000e710 con 0x7f15a0003a60 2024-08-15T17:58:13.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b000e870 con 0x7f15a0003a60 2024-08-15T17:58:13.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.878+0000 7f15b9107700 1 -- 172.21.15.104:0/616480278 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b000ec30 con 0x7f15a0003a60 2024-08-15T17:58:13.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 -- 172.21.15.104:0/616480278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15a0003a60 msgr2=0x7f15a0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 --2- 172.21.15.104:0/616480278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15a0003a60 0x7f15a0003e70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009c00 tx=0x7f15b00091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 -- 172.21.15.104:0/616480278 shutdown_connections 2024-08-15T17:58:13.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 --2- 172.21.15.104:0/616480278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15a0003a60 0x7f15a0003e70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 -- 172.21.15.104:0/616480278 >> 172.21.15.104:0/616480278 conn(0x7f15b40c9bf0 msgr2=0x7f15b406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:13.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 -- 172.21.15.104:0/616480278 shutdown_connections 2024-08-15T17:58:13.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 -- 172.21.15.104:0/616480278 wait complete. 2024-08-15T17:58:13.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 Processor -- start 2024-08-15T17:58:13.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 -- start start 2024-08-15T17:58:13.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0003a60 0x7f15b40cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15b40cb180 0x7f15b414b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b414b9c0 0x7f15b4148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:13.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b414c1d0 con 0x7f15b414b9c0 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b414bef0 con 0x7f15a0003a60 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15bc36d700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b414c530 con 0x7f15b40cb180 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15ba109700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0003a60 0x7f15b40cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15b9908700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15b40cb180 0x7f15b414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15ba109700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0003a60 0x7f15b40cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53028/0 (socket says 172.21.15.104:53028) 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15ba109700 1 -- 172.21.15.104:0/1232361442 learned_addr learned my addr 172.21.15.104:0/1232361442 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.882+0000 7f15ba90a700 1 --2- 172.21.15.104:0/1232361442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b414b9c0 0x7f15b4148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15ba109700 1 -- 172.21.15.104:0/1232361442 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15b40cb180 msgr2=0x7f15b414b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15ba109700 1 --2- 172.21.15.104:0/1232361442 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15b40cb180 0x7f15b414b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15ba109700 1 -- 172.21.15.104:0/1232361442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b414b9c0 msgr2=0x7f15b4148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15ba109700 1 --2- 172.21.15.104:0/1232361442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b414b9c0 0x7f15b4148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15ba109700 1 -- 172.21.15.104:0/1232361442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b414c810 con 0x7f15a0003a60 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15ba109700 1 --2- 172.21.15.104:0/1232361442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0003a60 0x7f15b40cac40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f15a000c420 tx=0x7f15b001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15aa7fc700 1 -- 172.21.15.104:0/1232361442 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0022070 con 0x7f15a0003a60 2024-08-15T17:58:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15aa7fc700 1 -- 172.21.15.104:0/1232361442 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b000e9e0 con 0x7f15a0003a60 2024-08-15T17:58:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15bc36d700 1 -- 172.21.15.104:0/1232361442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b414caf0 con 0x7f15a0003a60 2024-08-15T17:58:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15aa7fc700 1 -- 172.21.15.104:0/1232361442 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b001c450 con 0x7f15a0003a60 2024-08-15T17:58:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15bc36d700 1 -- 172.21.15.104:0/1232361442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b4149ec0 con 0x7f15a0003a60 2024-08-15T17:58:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15aa7fc700 1 -- 172.21.15.104:0/1232361442 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f15b001c880 con 0x7f15a0003a60 2024-08-15T17:58:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15aa7fc700 1 -- 172.21.15.104:0/1232361442 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f15b0030ab0 con 0x7f15a0003a60 2024-08-15T17:58:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.886+0000 7f15bc36d700 1 -- 172.21.15.104:0/1232361442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b414c390 con 0x7f15a0003a60 2024-08-15T17:58:13.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.890+0000 7f15aa7fc700 1 -- 172.21.15.104:0/1232361442 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f15b414c390 con 0x7f15a0003a60 2024-08-15T17:58:13.973 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.970+0000 7f15bc36d700 1 -- 172.21.15.104:0/1232361442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ea9c9c4a-9662-4a09-b81f-260ccc5c27d4", "id": 7} v 0) v1 -- 0x7f15b414bef0 con 0x7f15a0003a60 2024-08-15T17:58:13.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.978+0000 7f15aa7fc700 1 -- 172.21.15.104:0/1232361442 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ea9c9c4a-9662-4a09-b81f-260ccc5c27d4", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f15b0035060 con 0x7f15a0003a60 2024-08-15T17:58:13.981 INFO:teuthology.orchestra.run.smithi104.stdout:7 2024-08-15T17:58:13.983 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.978+0000 7f15bc36d700 1 -- 172.21.15.104:0/1232361442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0003a60 msgr2=0x7f15b40cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:13.983 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.978+0000 7f15bc36d700 1 --2- 172.21.15.104:0/1232361442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0003a60 0x7f15b40cac40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f15a000c420 tx=0x7f15b001a000 comp rx=0 tx=0).stop 2024-08-15T17:58:13.983 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.978+0000 7f15bc36d700 1 -- 172.21.15.104:0/1232361442 shutdown_connections 2024-08-15T17:58:13.983 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.978+0000 7f15bc36d700 1 --2- 172.21.15.104:0/1232361442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b414b9c0 0x7f15b4148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.983 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.978+0000 7f15bc36d700 1 --2- 172.21.15.104:0/1232361442 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15b40cb180 0x7f15b414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.983 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.978+0000 7f15bc36d700 1 --2- 172.21.15.104:0/1232361442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0003a60 0x7f15b40cac40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:13.983 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.978+0000 7f15bc36d700 1 -- 172.21.15.104:0/1232361442 >> 172.21.15.104:0/1232361442 conn(0x7f15b40c9bf0 msgr2=0x7f15b4067690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:13.983 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.978+0000 7f15bc36d700 1 -- 172.21.15.104:0/1232361442 shutdown_connections 2024-08-15T17:58:13.983 INFO:teuthology.orchestra.run.smithi104.stderr:2024-08-15T17:58:13.978+0000 7f15bc36d700 1 -- 172.21.15.104:0/1232361442 wait complete. 2024-08-15T17:58:13.994 INFO:tasks.ceph.osd.0:Restarting daemon 2024-08-15T17:58:13.994 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-08-15T17:58:13.997 INFO:tasks.ceph.osd.0:Started 2024-08-15T17:58:13.997 INFO:tasks.ceph.osd.1:Restarting daemon 2024-08-15T17:58:13.997 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-08-15T17:58:13.999 INFO:tasks.ceph.osd.1:Started 2024-08-15T17:58:13.999 INFO:tasks.ceph.osd.2:Restarting daemon 2024-08-15T17:58:13.999 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-08-15T17:58:14.001 INFO:tasks.ceph.osd.2:Started 2024-08-15T17:58:14.002 INFO:tasks.ceph.osd.3:Restarting daemon 2024-08-15T17:58:14.002 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-08-15T17:58:14.005 INFO:tasks.ceph.osd.3:Started 2024-08-15T17:58:14.005 INFO:tasks.ceph.osd.4:Restarting daemon 2024-08-15T17:58:14.005 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-08-15T17:58:14.007 INFO:tasks.ceph.osd.4:Started 2024-08-15T17:58:14.007 INFO:tasks.ceph.osd.5:Restarting daemon 2024-08-15T17:58:14.008 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-08-15T17:58:14.012 INFO:tasks.ceph.osd.5:Started 2024-08-15T17:58:14.012 INFO:tasks.ceph.osd.6:Restarting daemon 2024-08-15T17:58:14.012 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-08-15T17:58:14.015 INFO:tasks.ceph.osd.6:Started 2024-08-15T17:58:14.015 INFO:tasks.ceph.osd.7:Restarting daemon 2024-08-15T17:58:14.015 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-08-15T17:58:14.018 INFO:tasks.ceph.osd.7:Started 2024-08-15T17:58:14.019 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:14.064 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:14.058+0000 7fb9acbbc000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-15T17:58:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.131+0000 7f5961911700 1 Processor -- start 2024-08-15T17:58:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.131+0000 7f5961911700 1 -- start start 2024-08-15T17:58:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.131+0000 7f5961911700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f595c159240 0x7f595c0af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.131+0000 7f5961911700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c159610 0x7f595c0686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.131+0000 7f5961911700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595c0c7dc0 con 0x7f595c0c6c80 2024-08-15T17:58:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.131+0000 7f5961911700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595c06c170 con 0x7f595c159240 2024-08-15T17:58:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.131+0000 7f5961911700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c0c71b0 con 0x7f595c159610 2024-08-15T17:58:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.131+0000 7f5953fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f595c0c6c80 0x7f595c06ba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33994/0 (socket says 172.21.15.38:33994) 2024-08-15T17:58:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.131+0000 7f5953fff700 1 -- 172.21.15.38:0/3864109458 learned_addr learned my addr 172.21.15.38:0/3864109458 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.131+0000 7f595affd700 1 --2- 172.21.15.38:0/3864109458 >> v2:172.21.15.104:3300/0 conn(0x7f595c159240 0x7f595c0af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.131+0000 7f595b7fe700 1 --2- 172.21.15.38:0/3864109458 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c159610 0x7f595c0686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f595b7fe700 1 -- 172.21.15.38:0/3864109458 >> v2:172.21.15.104:3300/0 conn(0x7f595c159240 msgr2=0x7f595c0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f595b7fe700 1 --2- 172.21.15.38:0/3864109458 >> v2:172.21.15.104:3300/0 conn(0x7f595c159240 0x7f595c0af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f595b7fe700 1 -- 172.21.15.38:0/3864109458 >> v1:172.21.15.38:6789/0 conn(0x7f595c0c6c80 legacy=0x7f595c06ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f595b7fe700 1 -- 172.21.15.38:0/3864109458 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c0c7490 con 0x7f595c159610 2024-08-15T17:58:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f595affd700 1 --2- 172.21.15.38:0/3864109458 >> v2:172.21.15.104:3300/0 conn(0x7f595c159240 0x7f595c0af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f595a7fc700 1 -- 172.21.15.38:0/3864109458 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131174956 0 0) 0x7f595c0c7dc0 con 0x7f595c0c6c80 2024-08-15T17:58:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f595b7fe700 1 --2- 172.21.15.38:0/3864109458 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c159610 0x7f595c0686a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5954009e60 tx=0x7f5954009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c8f42f4f0e5b4b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f595a7fc700 1 -- 172.21.15.38:0/3864109458 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595400e070 con 0x7f595c159610 2024-08-15T17:58:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f595a7fc700 1 -- 172.21.15.38:0/3864109458 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595400bb60 con 0x7f595c159610 2024-08-15T17:58:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f595a7fc700 1 -- 172.21.15.38:0/3864109458 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954012430 con 0x7f595c159610 2024-08-15T17:58:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f5961911700 1 -- 172.21.15.38:0/3864109458 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c159610 msgr2=0x7f595c0686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f5961911700 1 --2- 172.21.15.38:0/3864109458 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c159610 0x7f595c0686a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5954009e60 tx=0x7f5954009780 comp rx=0 tx=0).stop 2024-08-15T17:58:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f5961911700 1 -- 172.21.15.38:0/3864109458 shutdown_connections 2024-08-15T17:58:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f5961911700 1 --2- 172.21.15.38:0/3864109458 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c159610 0x7f595c0686a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f5961911700 1 --2- 172.21.15.38:0/3864109458 >> v2:172.21.15.104:3300/0 conn(0x7f595c159240 0x7f595c0af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f5961911700 1 -- 172.21.15.38:0/3864109458 >> 172.21.15.38:0/3864109458 conn(0x7f595c0c9c20 msgr2=0x7f595c06d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f5961911700 1 -- 172.21.15.38:0/3864109458 shutdown_connections 2024-08-15T17:58:14.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f5961911700 1 -- 172.21.15.38:0/3864109458 wait complete. 2024-08-15T17:58:14.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f5961911700 1 Processor -- start 2024-08-15T17:58:14.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.135+0000 7f5961911700 1 -- start start 2024-08-15T17:58:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5961911700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c0c6c80 0x7f595c0a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5961911700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f595c159240 0x7f595c14b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5961911700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595c159610 0x7f595c0c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5961911700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c0a6860 con 0x7f595c159240 2024-08-15T17:58:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5961911700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c0a57c0 con 0x7f595c159610 2024-08-15T17:58:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5961911700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c0a5aa0 con 0x7f595c0c6c80 2024-08-15T17:58:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f595affd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c0c6c80 0x7f595c0a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5953fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f595c159240 0x7f595c14b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f595affd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c0c6c80 0x7f595c0a7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:49570/0 (socket says 172.21.15.38:49570) 2024-08-15T17:58:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f595affd700 1 -- 172.21.15.38:0/4217921246 learned_addr learned my addr 172.21.15.38:0/4217921246 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f595b7fe700 1 --2- 172.21.15.38:0/4217921246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595c159610 0x7f595c0c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f595affd700 1 -- 172.21.15.38:0/4217921246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595c159610 msgr2=0x7f595c0c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f595affd700 1 --2- 172.21.15.38:0/4217921246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595c159610 0x7f595c0c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f595affd700 1 -- 172.21.15.38:0/4217921246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f595c159240 msgr2=0x7f595c14b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f595affd700 1 --2- 172.21.15.38:0/4217921246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f595c159240 0x7f595c14b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f595affd700 1 -- 172.21.15.38:0/4217921246 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c009420 con 0x7f595c0c6c80 2024-08-15T17:58:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f595affd700 1 --2- 172.21.15.38:0/4217921246 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c0c6c80 0x7f595c0a7160 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f594c00ec80 tx=0x7f594c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5952ffd700 1 -- 172.21.15.38:0/4217921246 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00c920 con 0x7f595c0c6c80 2024-08-15T17:58:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5952ffd700 1 -- 172.21.15.38:0/4217921246 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c00ca80 con 0x7f595c0c6c80 2024-08-15T17:58:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5952ffd700 1 -- 172.21.15.38:0/4217921246 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00ce00 con 0x7f595c0c6c80 2024-08-15T17:58:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5961911700 1 -- 172.21.15.38:0/4217921246 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5954009a20 con 0x7f595c0c6c80 2024-08-15T17:58:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5961911700 1 -- 172.21.15.38:0/4217921246 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c0a6060 con 0x7f595c0c6c80 2024-08-15T17:58:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5952ffd700 1 -- 172.21.15.38:0/4217921246 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f594c005a00 con 0x7f595c0c6c80 2024-08-15T17:58:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.139+0000 7f5952ffd700 1 -- 172.21.15.38:0/4217921246 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f594c005cb0 con 0x7f595c0c6c80 2024-08-15T17:58:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.147+0000 7f5961911700 1 -- 172.21.15.38:0/4217921246 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595c0645c0 con 0x7f595c0c6c80 2024-08-15T17:58:14.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.151+0000 7f5952ffd700 1 -- 172.21.15.38:0/4217921246 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f594c021060 con 0x7f595c0c6c80 2024-08-15T17:58:14.179 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:14.174+0000 7fb9acbbc000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-15T17:58:14.189 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:14.187+0000 7f5fa5693000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-15T17:58:14.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.219+0000 7f5961911700 1 -- 172.21.15.38:0/4217921246 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f595c0a76a0 con 0x7f595c0c6c80 2024-08-15T17:58:14.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.219+0000 7f5952ffd700 1 -- 172.21.15.38:0/4217921246 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f594c029020 con 0x7f595c0c6c80 2024-08-15T17:58:14.220 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:14.220 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":10,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:13.978121+0000","last_up_change":"0.000000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:14.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.219+0000 7f5961911700 1 -- 172.21.15.38:0/4217921246 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c0c6c80 msgr2=0x7f595c0a7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:14.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.219+0000 7f5961911700 1 --2- 172.21.15.38:0/4217921246 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c0c6c80 0x7f595c0a7160 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f594c00ec80 tx=0x7f594c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:14.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.219+0000 7f5961911700 1 -- 172.21.15.38:0/4217921246 shutdown_connections 2024-08-15T17:58:14.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.219+0000 7f5961911700 1 --2- 172.21.15.38:0/4217921246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595c159610 0x7f595c0c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:14.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.219+0000 7f5961911700 1 --2- 172.21.15.38:0/4217921246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f595c159240 0x7f595c14b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:14.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.219+0000 7f5961911700 1 --2- 172.21.15.38:0/4217921246 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c0c6c80 0x7f595c0a7160 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:14.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.219+0000 7f5961911700 1 -- 172.21.15.38:0/4217921246 >> 172.21.15.38:0/4217921246 conn(0x7f595c0c9c20 msgr2=0x7f595c0b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:14.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.219+0000 7f5961911700 1 -- 172.21.15.38:0/4217921246 shutdown_connections 2024-08-15T17:58:14.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:14.219+0000 7f5961911700 1 -- 172.21.15.38:0/4217921246 wait complete. 2024-08-15T17:58:14.233 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-08-15T17:58:14.233 INFO:tasks.ceph:Waiting for OSDs to come up 2024-08-15T17:58:14.314 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:14.311+0000 7f5fa5693000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-15T17:58:14.370 INFO:tasks.ceph.osd.5.smithi104.stderr:2024-08-15T17:58:14.366+0000 7f9b67c7b3c0 -1 Falling back to public interface 2024-08-15T17:58:14.374 INFO:tasks.ceph.osd.6.smithi104.stderr:2024-08-15T17:58:14.370+0000 7f950319f3c0 -1 Falling back to public interface 2024-08-15T17:58:14.376 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-08-15T17:58:14.375+0000 7f41dc2fd3c0 -1 Falling back to public interface 2024-08-15T17:58:14.391 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-08-15T17:58:14.391+0000 7f91d898b3c0 -1 Falling back to public interface 2024-08-15T17:58:14.397 INFO:tasks.ceph.osd.7.smithi104.stderr:2024-08-15T17:58:14.394+0000 7f5f8a1243c0 -1 Falling back to public interface 2024-08-15T17:58:14.400 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-08-15T17:58:14.399+0000 7f1ebba033c0 -1 Falling back to public interface 2024-08-15T17:58:14.470 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:14.466+0000 7fb9acbbc000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-15T17:58:14.626 INFO:tasks.ceph.osd.4.smithi104.stderr:2024-08-15T17:58:14.622+0000 7feebc4243c0 -1 Falling back to public interface 2024-08-15T17:58:14.632 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:14.626+0000 7fb9acbbc000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-15T17:58:14.644 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-08-15T17:58:14.643+0000 7f2ec7fb03c0 -1 Falling back to public interface 2024-08-15T17:58:14.647 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:14.647+0000 7f5fa5693000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-15T17:58:14.700 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:14.694+0000 7fb9acbbc000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-15T17:58:14.752 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:14.746+0000 7fb9acbbc000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-15T17:58:14.809 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:14.806+0000 7fb9acbbc000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-15T17:58:14.811 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:14.811+0000 7f5fa5693000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-15T17:58:14.882 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:14.879+0000 7f5fa5693000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-15T17:58:14.935 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:14.935+0000 7f5fa5693000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-15T17:58:14.981 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:14.978+0000 7fb9acbbc000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-15T17:58:14.995 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:14.995+0000 7f5fa5693000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-15T17:58:15.039 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:15.134 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.130+0000 7fc3b7fff700 1 Processor -- start 2024-08-15T17:58:15.134 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.130+0000 7fc3b7fff700 1 -- start start 2024-08-15T17:58:15.135 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc3b806ace0 0x7fc3b8161050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:15.135 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b8157ab0 con 0x7fc3b8155ef0 2024-08-15T17:58:15.135 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b80678d0 con 0x7fc3b81562c0 2024-08-15T17:58:15.135 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b8068170 con 0x7fc3b806ace0 2024-08-15T17:58:15.135 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b6ffd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc3b8155ef0 0x7fc3b8066e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34002/0 (socket says 172.21.15.38:34002) 2024-08-15T17:58:15.135 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b6ffd700 1 -- 172.21.15.38:0/1140392426 learned_addr learned my addr 172.21.15.38:0/1140392426 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:15.136 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b77fe700 1 --2- 172.21.15.38:0/1140392426 >> v2:172.21.15.38:3300/0 conn(0x7fc3b806ace0 0x7fc3b8161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:15.136 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b67fc700 1 -- 172.21.15.38:0/1140392426 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971845308 0 0) 0x7fc3b8157ab0 con 0x7fc3b8155ef0 2024-08-15T17:58:15.136 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b67fc700 1 -- 172.21.15.38:0/1140392426 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3a4003570 con 0x7fc3b8155ef0 2024-08-15T17:58:15.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b67fc700 1 -- 172.21.15.38:0/1140392426 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546431837 0 0) 0x7fc3b80678d0 con 0x7fc3b81562c0 2024-08-15T17:58:15.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b77fe700 1 -- 172.21.15.38:0/1140392426 >> v1:172.21.15.104:6789/0 conn(0x7fc3b81562c0 legacy=0x7fc3b806a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:15.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b77fe700 1 -- 172.21.15.38:0/1140392426 >> v1:172.21.15.38:6789/0 conn(0x7fc3b8155ef0 legacy=0x7fc3b8066e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:15.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b77fe700 1 -- 172.21.15.38:0/1140392426 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b8067bb0 con 0x7fc3b806ace0 2024-08-15T17:58:15.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b77fe700 1 --2- 172.21.15.38:0/1140392426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b806ace0 0x7fc3b8161050 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009e60 tx=0x7fc3b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448f331a988ca215 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:15.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b67fc700 1 -- 172.21.15.38:0/1140392426 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b000e070 con 0x7fc3b806ace0 2024-08-15T17:58:15.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b67fc700 1 -- 172.21.15.38:0/1140392426 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b000b850 con 0x7fc3b806ace0 2024-08-15T17:58:15.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b67fc700 1 -- 172.21.15.38:0/1140392426 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8001470 con 0x7fc3b806ace0 2024-08-15T17:58:15.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 -- 172.21.15.38:0/1140392426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b806ace0 msgr2=0x7fc3b8161050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:15.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 --2- 172.21.15.38:0/1140392426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b806ace0 0x7fc3b8161050 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009e60 tx=0x7fc3b0009780 comp rx=0 tx=0).stop 2024-08-15T17:58:15.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 -- 172.21.15.38:0/1140392426 shutdown_connections 2024-08-15T17:58:15.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 --2- 172.21.15.38:0/1140392426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b806ace0 0x7fc3b8161050 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:15.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 -- 172.21.15.38:0/1140392426 >> 172.21.15.38:0/1140392426 conn(0x7fc3b806d1a0 msgr2=0x7fc3b806d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:15.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 -- 172.21.15.38:0/1140392426 shutdown_connections 2024-08-15T17:58:15.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 -- 172.21.15.38:0/1140392426 wait complete. 2024-08-15T17:58:15.140 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.134+0000 7fc3b7fff700 1 Processor -- start 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b7fff700 1 -- start start 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3b806ace0 0x7fc3b8139a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b8155ef0 0x7fc3b80ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b7fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3b81562c0 0x7fc3b8151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b7fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b8155200 con 0x7fc3b8155ef0 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b7fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b815adf0 con 0x7fc3b806ace0 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b7fff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b815b0d0 con 0x7fc3b81562c0 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3ae7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b8155ef0 0x7fc3b80ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3ae7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b8155ef0 0x7fc3b80ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37768/0 (socket says 172.21.15.38:37768) 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3ae7fc700 1 -- 172.21.15.38:0/450305041 learned_addr learned my addr 172.21.15.38:0/450305041 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b77fe700 1 --2- 172.21.15.38:0/450305041 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3b81562c0 0x7fc3b8151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:15.141 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b6ffd700 1 --2- 172.21.15.38:0/450305041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3b806ace0 0x7fc3b8139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3ae7fc700 1 -- 172.21.15.38:0/450305041 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3b81562c0 msgr2=0x7fc3b8151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3ae7fc700 1 --2- 172.21.15.38:0/450305041 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3b81562c0 0x7fc3b8151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3ae7fc700 1 -- 172.21.15.38:0/450305041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3b806ace0 msgr2=0x7fc3b8139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3ae7fc700 1 --2- 172.21.15.38:0/450305041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3b806ace0 0x7fc3b8139a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3ae7fc700 1 -- 172.21.15.38:0/450305041 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b0009a20 con 0x7fc3b8155ef0 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3ae7fc700 1 --2- 172.21.15.38:0/450305041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b8155ef0 0x7fc3b80ca450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000c530 tx=0x7fc3a000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3af7fe700 1 -- 172.21.15.38:0/450305041 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0011590 con 0x7fc3b8155ef0 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3af7fe700 1 -- 172.21.15.38:0/450305041 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a0011710 con 0x7fc3b8155ef0 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b7fff700 1 -- 172.21.15.38:0/450305041 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b815b3b0 con 0x7fc3b8155ef0 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3af7fe700 1 -- 172.21.15.38:0/450305041 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0011ad0 con 0x7fc3b8155ef0 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b7fff700 1 -- 172.21.15.38:0/450305041 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b8139f60 con 0x7fc3b8155ef0 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3af7fe700 1 -- 172.21.15.38:0/450305041 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fc3a001c440 con 0x7fc3b8155ef0 2024-08-15T17:58:15.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3af7fe700 1 -- 172.21.15.38:0/450305041 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fc3a0012b00 con 0x7fc3b8155ef0 2024-08-15T17:58:15.143 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.138+0000 7fc3b7fff700 1 -- 172.21.15.38:0/450305041 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b815b8b0 con 0x7fc3b8155ef0 2024-08-15T17:58:15.144 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:15.138+0000 7fb9acbbc000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-15T17:58:15.145 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.142+0000 7fc3af7fe700 1 -- 172.21.15.38:0/450305041 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fc3a0017070 con 0x7fc3b8155ef0 2024-08-15T17:58:15.156 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:15.154+0000 7f5fa5693000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-15T17:58:15.208 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:15.202+0000 7fb9acbbc000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-15T17:58:15.210 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.210+0000 7fc3b7fff700 1 -- 172.21.15.38:0/450305041 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3b8064ba0 con 0x7fc3b8155ef0 2024-08-15T17:58:15.211 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.210+0000 7fc3af7fe700 1 -- 172.21.15.38:0/450305041 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fc3a001c8d0 con 0x7fc3b8155ef0 2024-08-15T17:58:15.211 INFO:teuthology.misc.health.smithi038.stdout: 2024-08-15T17:58:15.211 INFO:teuthology.misc.health.smithi038.stdout:{"epoch":10,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:13.978121+0000","last_up_change":"0.000000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:15.212 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.210+0000 7fc3b7fff700 1 -- 172.21.15.38:0/450305041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b8155ef0 msgr2=0x7fc3b80ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:15.213 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.210+0000 7fc3b7fff700 1 --2- 172.21.15.38:0/450305041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b8155ef0 0x7fc3b80ca450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000c530 tx=0x7fc3a000c840 comp rx=0 tx=0).stop 2024-08-15T17:58:15.213 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.210+0000 7fc3b7fff700 1 -- 172.21.15.38:0/450305041 shutdown_connections 2024-08-15T17:58:15.213 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.210+0000 7fc3b7fff700 1 --2- 172.21.15.38:0/450305041 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3b81562c0 0x7fc3b8151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:15.213 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.210+0000 7fc3b7fff700 1 --2- 172.21.15.38:0/450305041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b8155ef0 0x7fc3b80ca450 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:15.213 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.210+0000 7fc3b7fff700 1 --2- 172.21.15.38:0/450305041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3b806ace0 0x7fc3b8139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:15.213 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.210+0000 7fc3b7fff700 1 -- 172.21.15.38:0/450305041 >> 172.21.15.38:0/450305041 conn(0x7fc3b806d1a0 msgr2=0x7fc3b806c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:15.213 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.210+0000 7fc3b7fff700 1 -- 172.21.15.38:0/450305041 shutdown_connections 2024-08-15T17:58:15.213 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:15.210+0000 7fc3b7fff700 1 -- 172.21.15.38:0/450305041 wait complete. 2024-08-15T17:58:15.222 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-08-15T17:58:15.324 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:15.322+0000 7f5fa5693000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-15T17:58:15.361 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:15.358+0000 7fb9acbbc000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-15T17:58:15.386 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:15.386+0000 7f5fa5693000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-15T17:58:15.427 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:15.422+0000 7fb9acbbc000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-15T17:58:15.549 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:15.546+0000 7f5fa5693000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-15T17:58:15.619 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:15.618+0000 7f5fa5693000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-15T17:58:16.131 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:16.126+0000 7fb9acbbc000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-15T17:58:16.359 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-08-15T17:58:16.358+0000 7f5fa5693000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-15T17:58:17.181 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:17.178+0000 7fb995c34700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-15T17:58:17.181 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-08-15T17:58:17.178+0000 7fb995c34700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-15T17:58:18.080 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-08-15T17:58:18.078+0000 7f91d898b3c0 -1 osd.0 0 log_to_monitors true 2024-08-15T17:58:18.081 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-08-15T17:58:18.078+0000 7f41dc2fd3c0 -1 osd.3 0 log_to_monitors true 2024-08-15T17:58:18.151 INFO:tasks.ceph.osd.5.smithi104.stderr:2024-08-15T17:58:18.146+0000 7f9b67c7b3c0 -1 osd.5 0 log_to_monitors true 2024-08-15T17:58:18.248 INFO:tasks.ceph.osd.6.smithi104.stderr:2024-08-15T17:58:18.246+0000 7f950319f3c0 -1 osd.6 0 log_to_monitors true 2024-08-15T17:58:18.251 INFO:tasks.ceph.osd.7.smithi104.stderr:2024-08-15T17:58:18.246+0000 7f5f8a1243c0 -1 osd.7 0 log_to_monitors true 2024-08-15T17:58:18.333 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-08-15T17:58:18.330+0000 7f1ebba033c0 -1 osd.1 0 log_to_monitors true 2024-08-15T17:58:18.363 INFO:tasks.ceph.osd.4.smithi104.stderr:2024-08-15T17:58:18.358+0000 7feebc4243c0 -1 osd.4 0 log_to_monitors true 2024-08-15T17:58:18.519 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-08-15T17:58:18.518+0000 7f2ec7fb03c0 -1 osd.2 0 log_to_monitors true 2024-08-15T17:58:20.887 INFO:tasks.ceph.osd.5.smithi104.stderr:2024-08-15T17:58:20.879+0000 7f9b64c40700 -1 osd.5 0 waiting for initial osdmap 2024-08-15T17:58:20.914 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-08-15T17:58:20.910+0000 7f91d5950700 -1 osd.0 0 waiting for initial osdmap 2024-08-15T17:58:20.920 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-08-15T17:58:20.918+0000 7f91ccece700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:58:20.922 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-08-15T17:58:20.918+0000 7f41d92c2700 -1 osd.3 0 waiting for initial osdmap 2024-08-15T17:58:20.928 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-08-15T17:58:20.926+0000 7f41d0840700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:58:20.968 INFO:tasks.ceph.osd.5.smithi104.stderr:2024-08-15T17:58:20.963+0000 7f9b5c1be700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:58:21.625 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-08-15T17:58:21.622+0000 7f2ec3750700 -1 osd.2 0 waiting for initial osdmap 2024-08-15T17:58:21.636 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-08-15T17:58:21.634+0000 7f2ebc4f3700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:58:21.683 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-08-15T17:58:21.682+0000 7f1eb71a3700 -1 osd.1 0 waiting for initial osdmap 2024-08-15T17:58:21.688 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-08-15T17:58:21.686+0000 7f1eaff46700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:58:21.990 INFO:tasks.ceph.osd.4.smithi104.stderr:2024-08-15T17:58:21.983+0000 7feeb7bc4700 -1 osd.4 0 waiting for initial osdmap 2024-08-15T17:58:21.998 INFO:tasks.ceph.osd.4.smithi104.stderr:2024-08-15T17:58:21.991+0000 7feeb0967700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:58:22.030 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:22.039 INFO:tasks.ceph.osd.6.smithi104.stderr:2024-08-15T17:58:22.031+0000 7f94fe93f700 -1 osd.6 0 waiting for initial osdmap 2024-08-15T17:58:22.048 INFO:tasks.ceph.osd.6.smithi104.stderr:2024-08-15T17:58:22.043+0000 7f94f76e2700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:58:22.134 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.130+0000 7fc638b7f700 1 Processor -- start 2024-08-15T17:58:22.134 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.130+0000 7fc638b7f700 1 -- start start 2024-08-15T17:58:22.135 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc638b7f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc634153b30 0x7fc6340ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:22.135 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc638b7f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc63415a570 0x7fc6340cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:22.135 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc638b7f700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6340cb120 con 0x7fc634153760 2024-08-15T17:58:22.135 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc638b7f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6340cb3c0 con 0x7fc634153b30 2024-08-15T17:58:22.136 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc638b7f700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6340cb6a0 con 0x7fc63415a570 2024-08-15T17:58:22.136 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc633fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc63415a570 0x7fc6340cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:22.136 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc633fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc63415a570 0x7fc6340cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:37868/0 (socket says 172.21.15.38:37868) 2024-08-15T17:58:22.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc633fff700 1 -- 172.21.15.38:0/2382213836 learned_addr learned my addr 172.21.15.38:0/2382213836 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:22.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc6337fe700 1 --1- 172.21.15.38:0/2382213836 >> v1:172.21.15.104:6789/0 conn(0x7fc634153760 0x7fc6340ca930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:47606/0 (socket says 172.21.15.38:47606) 2024-08-15T17:58:22.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc62bfff700 1 --2- 172.21.15.38:0/2382213836 >> v2:172.21.15.104:3300/0 conn(0x7fc634153b30 0x7fc6340ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:22.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc633fff700 1 -- 172.21.15.38:0/2382213836 >> v2:172.21.15.104:3300/0 conn(0x7fc634153b30 msgr2=0x7fc6340ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:22.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc633fff700 1 --2- 172.21.15.38:0/2382213836 >> v2:172.21.15.104:3300/0 conn(0x7fc634153b30 0x7fc6340ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:22.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc633fff700 1 -- 172.21.15.38:0/2382213836 >> v1:172.21.15.104:6789/0 conn(0x7fc634153760 legacy=0x7fc6340ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:22.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc633fff700 1 -- 172.21.15.38:0/2382213836 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6340cb980 con 0x7fc63415a570 2024-08-15T17:58:22.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc632ffd700 1 -- 172.21.15.38:0/2382213836 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986123771 0 0) 0x7fc6340cb120 con 0x7fc634153760 2024-08-15T17:58:22.137 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc633fff700 1 --2- 172.21.15.38:0/2382213836 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc63415a570 0x7fc6340cd0c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009f20 tx=0x7fc62c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f4478ad138d12f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:22.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc632ffd700 1 -- 172.21.15.38:0/2382213836 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c00e070 con 0x7fc63415a570 2024-08-15T17:58:22.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc632ffd700 1 -- 172.21.15.38:0/2382213836 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc62c00b9d0 con 0x7fc63415a570 2024-08-15T17:58:22.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc632ffd700 1 -- 172.21.15.38:0/2382213836 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0092e0 con 0x7fc63415a570 2024-08-15T17:58:22.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc638b7f700 1 -- 172.21.15.38:0/2382213836 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc63415a570 msgr2=0x7fc6340cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:22.138 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc638b7f700 1 --2- 172.21.15.38:0/2382213836 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc63415a570 0x7fc6340cd0c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009f20 tx=0x7fc62c009840 comp rx=0 tx=0).stop 2024-08-15T17:58:22.139 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc638b7f700 1 -- 172.21.15.38:0/2382213836 shutdown_connections 2024-08-15T17:58:22.139 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc638b7f700 1 --2- 172.21.15.38:0/2382213836 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc63415a570 0x7fc6340cd0c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:22.139 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc638b7f700 1 --2- 172.21.15.38:0/2382213836 >> v2:172.21.15.104:3300/0 conn(0x7fc634153b30 0x7fc6340ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:22.139 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.134+0000 7fc638b7f700 1 -- 172.21.15.38:0/2382213836 >> 172.21.15.38:0/2382213836 conn(0x7fc634063b90 msgr2=0x7fc6340994e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:22.139 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc638b7f700 1 -- 172.21.15.38:0/2382213836 shutdown_connections 2024-08-15T17:58:22.139 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc638b7f700 1 -- 172.21.15.38:0/2382213836 wait complete. 2024-08-15T17:58:22.140 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc638b7f700 1 Processor -- start 2024-08-15T17:58:22.140 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc638b7f700 1 -- start start 2024-08-15T17:58:22.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc638b7f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc634153760 0x7fc6340c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:22.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc638b7f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634153b30 0x7fc6340c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:22.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc638b7f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc63415a570 0x7fc63409b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:22.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc638b7f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6340c94d0 con 0x7fc634153b30 2024-08-15T17:58:22.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc638b7f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6340c9f70 con 0x7fc63415a570 2024-08-15T17:58:22.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc638b7f700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6340c88c0 con 0x7fc634153760 2024-08-15T17:58:22.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc633fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc63415a570 0x7fc63409b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:22.142 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc62bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634153b30 0x7fc6340c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:22.143 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc633fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc63415a570 0x7fc63409b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:36344/0 (socket says 172.21.15.38:36344) 2024-08-15T17:58:22.143 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc62bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634153b30 0x7fc6340c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42290/0 (socket says 172.21.15.38:42290) 2024-08-15T17:58:22.143 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc633fff700 1 -- 172.21.15.38:0/3338508396 learned_addr learned my addr 172.21.15.38:0/3338508396 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:22.143 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.138+0000 7fc6337fe700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc634153760 0x7fc6340c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:22.144 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62bfff700 1 -- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc634153760 msgr2=0x7fc6340c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:22.144 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62bfff700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc634153760 0x7fc6340c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:22.144 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62bfff700 1 -- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc63415a570 msgr2=0x7fc63409b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:22.144 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62bfff700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc63415a570 0x7fc63409b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:22.144 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62bfff700 1 -- 172.21.15.38:0/3338508396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c009ae0 con 0x7fc634153b30 2024-08-15T17:58:22.144 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62bfff700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634153b30 0x7fc6340c64e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00ea50 tx=0x7fc61c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:22.144 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62b7fe700 1 -- 172.21.15.38:0/3338508396 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00ee90 con 0x7fc634153b30 2024-08-15T17:58:22.145 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62b7fe700 1 -- 172.21.15.38:0/3338508396 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c00c7f0 con 0x7fc634153b30 2024-08-15T17:58:22.145 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc638b7f700 1 -- 172.21.15.38:0/3338508396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6340c8ba0 con 0x7fc634153b30 2024-08-15T17:58:22.145 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62b7fe700 1 -- 172.21.15.38:0/3338508396 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00cb90 con 0x7fc634153b30 2024-08-15T17:58:22.145 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc638b7f700 1 -- 172.21.15.38:0/3338508396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6340c9160 con 0x7fc634153b30 2024-08-15T17:58:22.145 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc638b7f700 1 -- 172.21.15.38:0/3338508396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc634064ba0 con 0x7fc634153b30 2024-08-15T17:58:22.145 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62b7fe700 1 -- 172.21.15.38:0/3338508396 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7fc61c013470 con 0x7fc634153b30 2024-08-15T17:58:22.146 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62b7fe700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6140607d0 0x7fc614062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:22.146 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.142+0000 7fc62b7fe700 1 -- 172.21.15.38:0/3338508396 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3627+0+0 (secure 0 0 0) 0x7fc61c07c450 con 0x7fc634153b30 2024-08-15T17:58:22.146 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.146+0000 7fc6337fe700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6140607d0 0x7fc614062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:22.147 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.146+0000 7fc6337fe700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6140607d0 0x7fc614062c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc624007fd0 tx=0x7fc624007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:22.151 INFO:tasks.ceph.osd.7.smithi104.stderr:2024-08-15T17:58:22.143+0000 7f5f870e9700 -1 osd.7 0 waiting for initial osdmap 2024-08-15T17:58:22.152 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.150+0000 7fc62b7fe700 1 -- 172.21.15.38:0/3338508396 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc61c04d1a0 con 0x7fc634153b30 2024-08-15T17:58:22.161 INFO:tasks.ceph.osd.7.smithi104.stderr:2024-08-15T17:58:22.155+0000 7f5f7e667700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:58:22.319 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.318+0000 7fc638b7f700 1 -- 172.21.15.38:0/3338508396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc63406f9f0 con 0x7fc634153b30 2024-08-15T17:58:22.320 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.318+0000 7fc62b7fe700 1 -- 172.21.15.38:0/3338508396 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+9118 (secure 0 0 0) 0x7fc61c0507c0 con 0x7fc634153b30 2024-08-15T17:58:22.321 INFO:teuthology.misc.health.smithi038.stdout: 2024-08-15T17:58:22.321 INFO:teuthology.misc.health.smithi038.stdout:{"epoch":14,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:21.902813+0000","last_up_change":"2024-08-15T17:58:21.902813+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:22.323 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 -- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6140607d0 msgr2=0x7fc614062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:22.323 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6140607d0 0x7fc614062c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc624007fd0 tx=0x7fc624007d50 comp rx=0 tx=0).stop 2024-08-15T17:58:22.323 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 -- 172.21.15.38:0/3338508396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634153b30 msgr2=0x7fc6340c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:22.323 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634153b30 0x7fc6340c64e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00ea50 tx=0x7fc61c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:22.323 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 -- 172.21.15.38:0/3338508396 shutdown_connections 2024-08-15T17:58:22.323 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6140607d0 0x7fc614062c80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:22.323 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc63415a570 0x7fc63409b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:22.323 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634153b30 0x7fc6340c64e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:22.324 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 --2- 172.21.15.38:0/3338508396 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc634153760 0x7fc6340c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:22.324 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 -- 172.21.15.38:0/3338508396 >> 172.21.15.38:0/3338508396 conn(0x7fc634063b90 msgr2=0x7fc6340ca520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:22.324 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 -- 172.21.15.38:0/3338508396 shutdown_connections 2024-08-15T17:58:22.324 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:22.322+0000 7fc638b7f700 1 -- 172.21.15.38:0/3338508396 wait complete. 2024-08-15T17:58:22.334 DEBUG:teuthology.misc:5 of 8 OSDs are up 2024-08-15T17:58:29.142 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:29.247 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee34c97700 1 Processor -- start 2024-08-15T17:58:29.247 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee34c97700 1 -- start start 2024-08-15T17:58:29.248 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee34c97700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fee3015dab0 0x7fee3006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.248 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee34c97700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fee3006c5d0 0x7fee300abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.249 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee34c97700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee300afe90 con 0x7fee3015d6e0 2024-08-15T17:58:29.249 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee34c97700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3006ca50 con 0x7fee3006c5d0 2024-08-15T17:58:29.249 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee34c97700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee300ad0c0 con 0x7fee3015dab0 2024-08-15T17:58:29.249 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2ffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fee3006c5d0 0x7fee300abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.249 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2ffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fee3006c5d0 0x7fee300abbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48488/0 (socket says 172.21.15.38:48488) 2024-08-15T17:58:29.249 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2ffff700 1 -- 172.21.15.38:0/2313617352 learned_addr learned my addr 172.21.15.38:0/2313617352 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:29.249 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2effd700 1 --2- 172.21.15.38:0/2313617352 >> v2:172.21.15.104:3300/0 conn(0x7fee3015dab0 0x7fee3006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.251 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2ffff700 1 -- 172.21.15.38:0/2313617352 >> v2:172.21.15.104:3300/0 conn(0x7fee3015dab0 msgr2=0x7fee3006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:29.251 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2ffff700 1 --2- 172.21.15.38:0/2313617352 >> v2:172.21.15.104:3300/0 conn(0x7fee3015dab0 0x7fee3006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.251 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2ffff700 1 -- 172.21.15.38:0/2313617352 >> v1:172.21.15.104:6789/0 conn(0x7fee3015d6e0 legacy=0x7fee30161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.251 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2ffff700 1 -- 172.21.15.38:0/2313617352 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee300af280 con 0x7fee3006c5d0 2024-08-15T17:58:29.251 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2ffff700 1 --2- 172.21.15.38:0/2313617352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3006c5d0 0x7fee300abbb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fee300ae950 tx=0x7fee28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad24895fbee6c95 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:29.251 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2e7fc700 1 -- 172.21.15.38:0/2313617352 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28009e70 con 0x7fee3006c5d0 2024-08-15T17:58:29.251 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2e7fc700 1 -- 172.21.15.38:0/2313617352 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2800b860 con 0x7fee3006c5d0 2024-08-15T17:58:29.251 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee2e7fc700 1 -- 172.21.15.38:0/2313617352 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2800bc20 con 0x7fee3006c5d0 2024-08-15T17:58:29.252 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee34c97700 1 -- 172.21.15.38:0/2313617352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3006c5d0 msgr2=0x7fee300abbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.252 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.246+0000 7fee34c97700 1 --2- 172.21.15.38:0/2313617352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3006c5d0 0x7fee300abbb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fee300ae950 tx=0x7fee28009250 comp rx=0 tx=0).stop 2024-08-15T17:58:29.252 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 -- 172.21.15.38:0/2313617352 shutdown_connections 2024-08-15T17:58:29.252 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 --2- 172.21.15.38:0/2313617352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3006c5d0 0x7fee300abbb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.252 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 --2- 172.21.15.38:0/2313617352 >> v2:172.21.15.104:3300/0 conn(0x7fee3015dab0 0x7fee3006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.252 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 -- 172.21.15.38:0/2313617352 >> 172.21.15.38:0/2313617352 conn(0x7fee30066e10 msgr2=0x7fee300700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:29.252 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 -- 172.21.15.38:0/2313617352 shutdown_connections 2024-08-15T17:58:29.252 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 -- 172.21.15.38:0/2313617352 wait complete. 2024-08-15T17:58:29.253 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 Processor -- start 2024-08-15T17:58:29.253 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 -- start start 2024-08-15T17:58:29.255 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee3006c5d0 0x7fee3016dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee3015d6e0 0x7fee30155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3015dab0 0x7fee30159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3015bcc0 con 0x7fee3015dab0 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3015a960 con 0x7fee3006c5d0 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.250+0000 7fee34c97700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3015b390 con 0x7fee3015d6e0 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee2f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee3006c5d0 0x7fee3016dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee2effd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee3015d6e0 0x7fee30155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee2f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee3006c5d0 0x7fee3016dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:51988/0 (socket says 172.21.15.38:51988) 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee2f7fe700 1 -- 172.21.15.38:0/4269978773 learned_addr learned my addr 172.21.15.38:0/4269978773 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee2ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3015dab0 0x7fee30159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee2ffff700 1 -- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee3015d6e0 msgr2=0x7fee30155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee2ffff700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee3015d6e0 0x7fee30155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee2ffff700 1 -- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee3006c5d0 msgr2=0x7fee3016dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.256 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee2ffff700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee3006c5d0 0x7fee3016dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.257 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee2ffff700 1 -- 172.21.15.38:0/4269978773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee280094f0 con 0x7fee3015dab0 2024-08-15T17:58:29.257 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee2ffff700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3015dab0 0x7fee30159ff0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fee30066810 tx=0x7fee28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:29.257 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee277fe700 1 -- 172.21.15.38:0/4269978773 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2801f070 con 0x7fee3015dab0 2024-08-15T17:58:29.257 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee277fe700 1 -- 172.21.15.38:0/4269978773 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2800b9d0 con 0x7fee3015dab0 2024-08-15T17:58:29.257 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee34c97700 1 -- 172.21.15.38:0/4269978773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3015b670 con 0x7fee3015dab0 2024-08-15T17:58:29.257 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee277fe700 1 -- 172.21.15.38:0/4269978773 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28011420 con 0x7fee3015dab0 2024-08-15T17:58:29.257 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee34c97700 1 -- 172.21.15.38:0/4269978773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee30156420 con 0x7fee3015dab0 2024-08-15T17:58:29.257 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.254+0000 7fee34c97700 1 -- 172.21.15.38:0/4269978773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3015be30 con 0x7fee3015dab0 2024-08-15T17:58:29.259 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.258+0000 7fee277fe700 1 -- 172.21.15.38:0/4269978773 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fee28011790 con 0x7fee3015dab0 2024-08-15T17:58:29.260 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.258+0000 7fee277fe700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fee10060860 0x7fee10062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.260 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.258+0000 7fee277fe700 1 -- 172.21.15.38:0/4269978773 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fee2807e170 con 0x7fee3015dab0 2024-08-15T17:58:29.260 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.258+0000 7fee2f7fe700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fee10060860 0x7fee10062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.261 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.258+0000 7fee2f7fe700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fee10060860 0x7fee10062d10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fee180078a0 tx=0x7fee18007830 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:29.265 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.262+0000 7fee277fe700 1 -- 172.21.15.38:0/4269978773 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fee2804e970 con 0x7fee3015dab0 2024-08-15T17:58:29.433 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.430+0000 7fee34c97700 1 -- 172.21.15.38:0/4269978773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee30064ba0 con 0x7fee3015dab0 2024-08-15T17:58:29.435 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee277fe700 1 -- 172.21.15.38:0/4269978773 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12127 (secure 0 0 0) 0x7fee28051f90 con 0x7fee3015dab0 2024-08-15T17:58:29.435 INFO:teuthology.misc.health.smithi038.stdout: 2024-08-15T17:58:29.435 INFO:teuthology.misc.health.smithi038.stdout:{"epoch":17,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:25.005981+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:29.438 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 -- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fee10060860 msgr2=0x7fee10062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.438 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fee10060860 0x7fee10062d10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fee180078a0 tx=0x7fee18007830 comp rx=0 tx=0).stop 2024-08-15T17:58:29.438 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 -- 172.21.15.38:0/4269978773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3015dab0 msgr2=0x7fee30159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.438 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3015dab0 0x7fee30159ff0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fee30066810 tx=0x7fee28017000 comp rx=0 tx=0).stop 2024-08-15T17:58:29.438 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 -- 172.21.15.38:0/4269978773 shutdown_connections 2024-08-15T17:58:29.438 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fee10060860 0x7fee10062d10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.438 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3015dab0 0x7fee30159ff0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.438 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee3015d6e0 0x7fee30155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.438 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 --2- 172.21.15.38:0/4269978773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee3006c5d0 0x7fee3016dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.439 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 -- 172.21.15.38:0/4269978773 >> 172.21.15.38:0/4269978773 conn(0x7fee30066e10 msgr2=0x7fee300c9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:29.439 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 -- 172.21.15.38:0/4269978773 shutdown_connections 2024-08-15T17:58:29.439 INFO:teuthology.misc.health.smithi038.stderr:2024-08-15T17:58:29.434+0000 7fee34c97700 1 -- 172.21.15.38:0/4269978773 wait complete. 2024-08-15T17:58:29.449 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-08-15T17:58:29.449 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-08-15T17:58:29.449 INFO:tasks.ceph.mds.a:Restarting daemon 2024-08-15T17:58:29.449 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-08-15T17:58:29.452 INFO:tasks.ceph.mds.a:Started 2024-08-15T17:58:29.452 INFO:tasks.ceph.mds.c:Restarting daemon 2024-08-15T17:58:29.452 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-08-15T17:58:29.454 INFO:tasks.ceph.mds.c:Started 2024-08-15T17:58:29.454 INFO:tasks.ceph.mds.b:Restarting daemon 2024-08-15T17:58:29.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-08-15T17:58:29.457 INFO:tasks.ceph.mds.b:Started 2024-08-15T17:58:29.457 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-15T17:58:29.457 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-15T17:58:29.459 INFO:tasks.ceph.mds.d:Started 2024-08-15T17:58:29.459 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-08-15T17:58:29.459 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:29.520 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-08-15T17:58:29.541 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-08-15T17:58:29.541 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-08-15T17:58:29.546 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-08-15T17:58:29.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa8a53700 1 Processor -- start 2024-08-15T17:58:29.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa8a53700 1 -- start start 2024-08-15T17:58:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa8a53700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7effa40af340 0x7effa4067840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa8a53700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7effa4067d80 0x7effa4157150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa8a53700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa40cc8c0 con 0x7effa4156d80 2024-08-15T17:58:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa8a53700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa40cbca0 con 0x7effa4067d80 2024-08-15T17:58:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa8a53700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa4068200 con 0x7effa40af340 2024-08-15T17:58:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa259c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7effa40af340 0x7effa4067840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa359e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7effa4067d80 0x7effa4157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa2d9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7effa4156d80 0x7effa40aec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:56888/0 (socket says 172.21.15.38:56888) 2024-08-15T17:58:29.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa2d9d700 1 -- 172.21.15.38:0/3040251187 learned_addr learned my addr 172.21.15.38:0/3040251187 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa359e700 1 -- 172.21.15.38:0/3040251187 >> v2:172.21.15.104:3300/0 conn(0x7effa40af340 msgr2=0x7effa4067840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa359e700 1 --2- 172.21.15.38:0/3040251187 >> v2:172.21.15.104:3300/0 conn(0x7effa40af340 0x7effa4067840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.570+0000 7effa359e700 1 -- 172.21.15.38:0/3040251187 >> v1:172.21.15.104:6789/0 conn(0x7effa4156d80 legacy=0x7effa40aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa359e700 1 -- 172.21.15.38:0/3040251187 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa4068450 con 0x7effa4067d80 2024-08-15T17:58:29.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa359e700 1 --2- 172.21.15.38:0/3040251187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effa4067d80 0x7effa4157150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7effa406d4c0 tx=0x7eff9c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9b0d5548a55aa1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:29.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa1d9b700 1 -- 172.21.15.38:0/3040251187 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00e070 con 0x7effa4067d80 2024-08-15T17:58:29.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa1d9b700 1 -- 172.21.15.38:0/3040251187 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c00bb60 con 0x7effa4067d80 2024-08-15T17:58:29.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa1d9b700 1 -- 172.21.15.38:0/3040251187 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c012430 con 0x7effa4067d80 2024-08-15T17:58:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa8a53700 1 -- 172.21.15.38:0/3040251187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effa4067d80 msgr2=0x7effa4157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa8a53700 1 --2- 172.21.15.38:0/3040251187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effa4067d80 0x7effa4157150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7effa406d4c0 tx=0x7eff9c009780 comp rx=0 tx=0).stop 2024-08-15T17:58:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa8a53700 1 -- 172.21.15.38:0/3040251187 shutdown_connections 2024-08-15T17:58:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa8a53700 1 --2- 172.21.15.38:0/3040251187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effa4067d80 0x7effa4157150 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa8a53700 1 --2- 172.21.15.38:0/3040251187 >> v2:172.21.15.104:3300/0 conn(0x7effa40af340 0x7effa4067840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa8a53700 1 -- 172.21.15.38:0/3040251187 >> 172.21.15.38:0/3040251187 conn(0x7effa406e070 msgr2=0x7effa40b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa8a53700 1 -- 172.21.15.38:0/3040251187 shutdown_connections 2024-08-15T17:58:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.574+0000 7effa8a53700 1 -- 172.21.15.38:0/3040251187 wait complete. 2024-08-15T17:58:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa8a53700 1 Processor -- start 2024-08-15T17:58:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa8a53700 1 -- start start 2024-08-15T17:58:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa8a53700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effa4067d80 0x7effa4155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa8a53700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effa40af340 0x7effa4150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa8a53700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7effa4156d80 0x7effa40a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa8a53700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa4159c90 con 0x7effa4067d80 2024-08-15T17:58:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa8a53700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa4159080 con 0x7effa40af340 2024-08-15T17:58:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa8a53700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa4159360 con 0x7effa4156d80 2024-08-15T17:58:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effa4067d80 0x7effa4155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effa4067d80 0x7effa4155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48542/0 (socket says 172.21.15.38:48542) 2024-08-15T17:58:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa2d9d700 1 -- 172.21.15.38:0/3044551909 learned_addr learned my addr 172.21.15.38:0/3044551909 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa259c700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effa40af340 0x7effa4150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa359e700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7effa4156d80 0x7effa40a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa2d9d700 1 -- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7effa4156d80 msgr2=0x7effa40a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa2d9d700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7effa4156d80 0x7effa40a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa2d9d700 1 -- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effa40af340 msgr2=0x7effa4150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa2d9d700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effa40af340 0x7effa4150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa2d9d700 1 -- 172.21.15.38:0/3044551909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c009a20 con 0x7effa4067d80 2024-08-15T17:58:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa2d9d700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effa4067d80 0x7effa4155fb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7eff9800c7c0 tx=0x7eff98007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7eff8affd700 1 -- 172.21.15.38:0/3044551909 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98005b30 con 0x7effa4067d80 2024-08-15T17:58:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa8a53700 1 -- 172.21.15.38:0/3044551909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa4159640 con 0x7effa4067d80 2024-08-15T17:58:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7eff8affd700 1 -- 172.21.15.38:0/3044551909 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff98005cb0 con 0x7effa4067d80 2024-08-15T17:58:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7eff8affd700 1 -- 172.21.15.38:0/3044551909 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98004a80 con 0x7effa4067d80 2024-08-15T17:58:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.578+0000 7effa8a53700 1 -- 172.21.15.38:0/3044551909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa41564f0 con 0x7effa4067d80 2024-08-15T17:58:29.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.582+0000 7effa8a53700 1 -- 172.21.15.38:0/3044551909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa40638f0 con 0x7effa4067d80 2024-08-15T17:58:29.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.582+0000 7eff8affd700 1 -- 172.21.15.38:0/3044551909 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7eff980070f0 con 0x7effa4067d80 2024-08-15T17:58:29.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.582+0000 7eff8affd700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff90060860 0x7eff90062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.582+0000 7eff8affd700 1 -- 172.21.15.38:0/3044551909 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7eff9807c130 con 0x7effa4067d80 2024-08-15T17:58:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.582+0000 7effa259c700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff90060860 0x7eff90062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.582+0000 7effa259c700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff90060860 0x7eff90062d10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7effa4071650 tx=0x7eff94008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:29.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.590+0000 7eff8affd700 1 -- 172.21.15.38:0/3044551909 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7eff9804c8b0 con 0x7effa4067d80 2024-08-15T17:58:29.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.750+0000 7effa8a53700 1 -- 172.21.15.38:0/3044551909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effa406ae60 con 0x7effa4067d80 2024-08-15T17:58:29.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.750+0000 7eff8affd700 1 -- 172.21.15.38:0/3044551909 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12127 (secure 0 0 0) 0x7eff98018090 con 0x7effa4067d80 2024-08-15T17:58:29.752 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:29.753 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:25.005981+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:29.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 -- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff90060860 msgr2=0x7eff90062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff90060860 0x7eff90062d10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7effa4071650 tx=0x7eff94008040 comp rx=0 tx=0).stop 2024-08-15T17:58:29.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 -- 172.21.15.38:0/3044551909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effa4067d80 msgr2=0x7effa4155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effa4067d80 0x7effa4155fb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7eff9800c7c0 tx=0x7eff98007f70 comp rx=0 tx=0).stop 2024-08-15T17:58:29.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 -- 172.21.15.38:0/3044551909 shutdown_connections 2024-08-15T17:58:29.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff90060860 0x7eff90062d10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7effa4156d80 0x7effa40a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effa40af340 0x7effa4150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 --2- 172.21.15.38:0/3044551909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effa4067d80 0x7effa4155fb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 -- 172.21.15.38:0/3044551909 >> 172.21.15.38:0/3044551909 conn(0x7effa406e070 msgr2=0x7effa4069b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:29.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 -- 172.21.15.38:0/3044551909 shutdown_connections 2024-08-15T17:58:29.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.754+0000 7effa8a53700 1 -- 172.21.15.38:0/3044551909 wait complete. 2024-08-15T17:58:29.767 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-15T17:58:29.767 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:58:29.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd2afd6700 1 Processor -- start 2024-08-15T17:58:29.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd2afd6700 1 -- start start 2024-08-15T17:58:29.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd2afd6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd240688d0 0x7fbd240aab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd2afd6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd24068ca0 0x7fbd24066ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd2afd6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbd24158b70 0x7fbd24158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd2afd6700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd24159530 con 0x7fbd24068ca0 2024-08-15T17:58:29.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd2afd6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd24067420 con 0x7fbd24158b70 2024-08-15T17:58:29.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd2afd6700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240ab0f0 con 0x7fbd240688d0 2024-08-15T17:58:29.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd297d3700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd24068ca0 0x7fbd24066ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd297d3700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd24068ca0 0x7fbd24066ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48548/0 (socket says 172.21.15.38:48548) 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd297d3700 1 -- 172.21.15.38:0/908121981 learned_addr learned my addr 172.21.15.38:0/908121981 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd2a7d5700 1 --2- 172.21.15.38:0/908121981 >> v2:172.21.15.104:3300/0 conn(0x7fbd24158b70 0x7fbd24158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd29fd4700 1 --2- 172.21.15.38:0/908121981 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd240688d0 0x7fbd240aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd297d3700 1 -- 172.21.15.38:0/908121981 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd240688d0 msgr2=0x7fbd240aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd297d3700 1 --2- 172.21.15.38:0/908121981 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd240688d0 0x7fbd240aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd297d3700 1 -- 172.21.15.38:0/908121981 >> v2:172.21.15.104:3300/0 conn(0x7fbd24158b70 msgr2=0x7fbd24158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd297d3700 1 --2- 172.21.15.38:0/908121981 >> v2:172.21.15.104:3300/0 conn(0x7fbd24158b70 0x7fbd24158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd297d3700 1 -- 172.21.15.38:0/908121981 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd240ab3d0 con 0x7fbd24068ca0 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd2a7d5700 1 --2- 172.21.15.38:0/908121981 >> v2:172.21.15.104:3300/0 conn(0x7fbd24158b70 0x7fbd24158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd29fd4700 1 --2- 172.21.15.38:0/908121981 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd240688d0 0x7fbd240aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:29.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd297d3700 1 --2- 172.21.15.38:0/908121981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd24068ca0 0x7fbd24066ee0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd240b2810 tx=0x7fbd20009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0dae44ba237984 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:29.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd28fd2700 1 -- 172.21.15.38:0/908121981 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000e070 con 0x7fbd24068ca0 2024-08-15T17:58:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd28fd2700 1 -- 172.21.15.38:0/908121981 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2000ba50 con 0x7fbd24068ca0 2024-08-15T17:58:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.874+0000 7fbd28fd2700 1 -- 172.21.15.38:0/908121981 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd200092e0 con 0x7fbd24068ca0 2024-08-15T17:58:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.878+0000 7fbd2afd6700 1 -- 172.21.15.38:0/908121981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd24068ca0 msgr2=0x7fbd24066ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.878+0000 7fbd2afd6700 1 --2- 172.21.15.38:0/908121981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd24068ca0 0x7fbd24066ee0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd240b2810 tx=0x7fbd20009840 comp rx=0 tx=0).stop 2024-08-15T17:58:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.878+0000 7fbd2afd6700 1 -- 172.21.15.38:0/908121981 shutdown_connections 2024-08-15T17:58:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.878+0000 7fbd2afd6700 1 --2- 172.21.15.38:0/908121981 >> v2:172.21.15.104:3300/0 conn(0x7fbd24158b70 0x7fbd24158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.878+0000 7fbd2afd6700 1 --2- 172.21.15.38:0/908121981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd24068ca0 0x7fbd24066ee0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.878+0000 7fbd2afd6700 1 --2- 172.21.15.38:0/908121981 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd240688d0 0x7fbd240aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.878+0000 7fbd2afd6700 1 -- 172.21.15.38:0/908121981 >> 172.21.15.38:0/908121981 conn(0x7fbd2406e310 msgr2=0x7fbd2406b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.878+0000 7fbd2afd6700 1 -- 172.21.15.38:0/908121981 shutdown_connections 2024-08-15T17:58:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.878+0000 7fbd2afd6700 1 -- 172.21.15.38:0/908121981 wait complete. 2024-08-15T17:58:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.878+0000 7fbd2afd6700 1 Processor -- start 2024-08-15T17:58:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd2afd6700 1 -- start start 2024-08-15T17:58:29.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd2afd6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd240688d0 0x7fbd2415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd2afd6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd24068ca0 0x7fbd2414a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd2afd6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd24158b70 0x7fbd240b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd2afd6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2414a950 con 0x7fbd240688d0 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd2afd6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240afa80 con 0x7fbd24158b70 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd2afd6700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240b2510 con 0x7fbd24068ca0 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd29fd4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd240688d0 0x7fbd2415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd29fd4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd240688d0 0x7fbd2415d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48558/0 (socket says 172.21.15.38:48558) 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd29fd4700 1 -- 172.21.15.38:0/3247564166 learned_addr learned my addr 172.21.15.38:0/3247564166 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd297d3700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd24068ca0 0x7fbd2414a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd2a7d5700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd24158b70 0x7fbd240b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd29fd4700 1 -- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd24068ca0 msgr2=0x7fbd2414a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd29fd4700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd24068ca0 0x7fbd2414a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd29fd4700 1 -- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd24158b70 msgr2=0x7fbd240b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd29fd4700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd24158b70 0x7fbd240b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd29fd4700 1 -- 172.21.15.38:0/3247564166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd20009ae0 con 0x7fbd240688d0 2024-08-15T17:58:29.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd29fd4700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd240688d0 0x7fbd2415d5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800e8d0 tx=0x7fbd18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:29.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd15ffb700 1 -- 172.21.15.38:0/3247564166 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18013410 con 0x7fbd240688d0 2024-08-15T17:58:29.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd15ffb700 1 -- 172.21.15.38:0/3247564166 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd18013570 con 0x7fbd240688d0 2024-08-15T17:58:29.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd2afd6700 1 -- 172.21.15.38:0/3247564166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2415dca0 con 0x7fbd240688d0 2024-08-15T17:58:29.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd15ffb700 1 -- 172.21.15.38:0/3247564166 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18013910 con 0x7fbd240688d0 2024-08-15T17:58:29.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.882+0000 7fbd2afd6700 1 -- 172.21.15.38:0/3247564166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2415e260 con 0x7fbd240688d0 2024-08-15T17:58:29.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.886+0000 7fbd2afd6700 1 -- 172.21.15.38:0/3247564166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd240638f0 con 0x7fbd240688d0 2024-08-15T17:58:29.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.886+0000 7fbd15ffb700 1 -- 172.21.15.38:0/3247564166 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbd18013d00 con 0x7fbd240688d0 2024-08-15T17:58:29.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.886+0000 7fbd15ffb700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbd08060890 0x7fbd08062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:29.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.886+0000 7fbd15ffb700 1 -- 172.21.15.38:0/3247564166 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbd1807c500 con 0x7fbd240688d0 2024-08-15T17:58:29.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.886+0000 7fbd297d3700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbd08060890 0x7fbd08062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.890+0000 7fbd297d3700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbd08060890 0x7fbd08062d40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009880 tx=0x7fbd20018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:29.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:29.894+0000 7fbd15ffb700 1 -- 172.21.15.38:0/3247564166 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbd1804cca0 con 0x7fbd240688d0 2024-08-15T17:58:30.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.062+0000 7fbd2afd6700 1 -- 172.21.15.38:0/3247564166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd240708e0 con 0x7fbd240688d0 2024-08-15T17:58:30.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.062+0000 7fbd15ffb700 1 -- 172.21.15.38:0/3247564166 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd180502c0 con 0x7fbd240688d0 2024-08-15T17:58:30.065 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T17:58:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 -- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbd08060890 msgr2=0x7fbd08062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbd08060890 0x7fbd08062d40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009880 tx=0x7fbd20018000 comp rx=0 tx=0).stop 2024-08-15T17:58:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 -- 172.21.15.38:0/3247564166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd240688d0 msgr2=0x7fbd2415d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd240688d0 0x7fbd2415d5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800e8d0 tx=0x7fbd18008040 comp rx=0 tx=0).stop 2024-08-15T17:58:30.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 -- 172.21.15.38:0/3247564166 shutdown_connections 2024-08-15T17:58:30.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbd08060890 0x7fbd08062d40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd24158b70 0x7fbd240b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd24068ca0 0x7fbd2414a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 --2- 172.21.15.38:0/3247564166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd240688d0 0x7fbd2415d5c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 -- 172.21.15.38:0/3247564166 >> 172.21.15.38:0/3247564166 conn(0x7fbd2406e310 msgr2=0x7fbd24069610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:30.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 -- 172.21.15.38:0/3247564166 shutdown_connections 2024-08-15T17:58:30.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.066+0000 7fbd2afd6700 1 -- 172.21.15.38:0/3247564166 wait complete. 2024-08-15T17:58:30.079 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:58:30.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.182+0000 7fdb2aa19700 1 Processor -- start 2024-08-15T17:58:30.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.182+0000 7fdb2aa19700 1 -- start start 2024-08-15T17:58:30.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdb2406e620 0x7fdb240b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:30.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdb240b1630 0x7fdb24141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:30.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb24141a00 con 0x7fdb240b0d20 2024-08-15T17:58:30.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb24140b10 con 0x7fdb2406e620 2024-08-15T17:58:30.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb24140df0 con 0x7fdb240b1630 2024-08-15T17:58:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb29a17700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdb240b0d20 0x7fdb2406df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50420/0 (socket says 172.21.15.38:50420) 2024-08-15T17:58:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb29a17700 1 -- 172.21.15.38:0/3012876480 learned_addr learned my addr 172.21.15.38:0/3012876480 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2a218700 1 --2- 172.21.15.38:0/3012876480 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdb240b1630 0x7fdb24141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb29216700 1 --2- 172.21.15.38:0/3012876480 >> v2:172.21.15.104:3300/0 conn(0x7fdb2406e620 0x7fdb240b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb28a15700 1 -- 172.21.15.38:0/3012876480 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731696702 0 0) 0x7fdb24141a00 con 0x7fdb240b0d20 2024-08-15T17:58:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb28a15700 1 -- 172.21.15.38:0/3012876480 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb10003570 con 0x7fdb240b0d20 2024-08-15T17:58:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb29216700 1 -- 172.21.15.38:0/3012876480 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdb240b1630 msgr2=0x7fdb24141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb29216700 1 --2- 172.21.15.38:0/3012876480 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdb240b1630 0x7fdb24141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb29216700 1 -- 172.21.15.38:0/3012876480 >> v1:172.21.15.38:6789/0 conn(0x7fdb240b0d20 legacy=0x7fdb2406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:30.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb29216700 1 -- 172.21.15.38:0/3012876480 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb241410d0 con 0x7fdb2406e620 2024-08-15T17:58:30.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2a218700 1 --2- 172.21.15.38:0/3012876480 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdb240b1630 0x7fdb24141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:30.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb29216700 1 --2- 172.21.15.38:0/3012876480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb2406e620 0x7fdb240b10f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdb180098e0 tx=0x7fdb180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50b2a0eebc350c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:30.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb28a15700 1 -- 172.21.15.38:0/3012876480 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb18011630 con 0x7fdb2406e620 2024-08-15T17:58:30.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb28a15700 1 -- 172.21.15.38:0/3012876480 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb18011790 con 0x7fdb2406e620 2024-08-15T17:58:30.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb28a15700 1 -- 172.21.15.38:0/3012876480 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb18011b10 con 0x7fdb2406e620 2024-08-15T17:58:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 -- 172.21.15.38:0/3012876480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb2406e620 msgr2=0x7fdb240b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 --2- 172.21.15.38:0/3012876480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb2406e620 0x7fdb240b10f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdb180098e0 tx=0x7fdb180091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 -- 172.21.15.38:0/3012876480 shutdown_connections 2024-08-15T17:58:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 --2- 172.21.15.38:0/3012876480 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdb240b1630 0x7fdb24141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 --2- 172.21.15.38:0/3012876480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb2406e620 0x7fdb240b10f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 -- 172.21.15.38:0/3012876480 >> 172.21.15.38:0/3012876480 conn(0x7fdb240678d0 msgr2=0x7fdb24067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 -- 172.21.15.38:0/3012876480 shutdown_connections 2024-08-15T17:58:30.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.186+0000 7fdb2aa19700 1 -- 172.21.15.38:0/3012876480 wait complete. 2024-08-15T17:58:30.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.190+0000 7fdb2aa19700 1 Processor -- start 2024-08-15T17:58:30.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.190+0000 7fdb2aa19700 1 -- start start 2024-08-15T17:58:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.190+0000 7fdb2aa19700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdb2406e620 0x7fdb24171b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.190+0000 7fdb2aa19700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb240b0d20 0x7fdb2415c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.190+0000 7fdb2aa19700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb240b1630 0x7fdb2413f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.190+0000 7fdb2aa19700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2416db60 con 0x7fdb240b1630 2024-08-15T17:58:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.190+0000 7fdb2aa19700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb24146b80 con 0x7fdb240b0d20 2024-08-15T17:58:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.190+0000 7fdb2aa19700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb24172070 con 0x7fdb2406e620 2024-08-15T17:58:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.190+0000 7fdb2a218700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb240b1630 0x7fdb2413f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb2a218700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb240b1630 0x7fdb2413f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48570/0 (socket says 172.21.15.38:48570) 2024-08-15T17:58:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb2a218700 1 -- 172.21.15.38:0/2298146278 learned_addr learned my addr 172.21.15.38:0/2298146278 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb29216700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb240b0d20 0x7fdb2415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb29a17700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdb2406e620 0x7fdb24171b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb2a218700 1 -- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdb2406e620 msgr2=0x7fdb24171b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb2a218700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdb2406e620 0x7fdb24171b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb2a218700 1 -- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb240b0d20 msgr2=0x7fdb2415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb2a218700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb240b0d20 0x7fdb2415c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb2a218700 1 -- 172.21.15.38:0/2298146278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1c009a20 con 0x7fdb240b1630 2024-08-15T17:58:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb29216700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb240b0d20 0x7fdb2415c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb2a218700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb240b1630 0x7fdb2413f310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009fc0 tx=0x7fdb1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb15ffb700 1 -- 172.21.15.38:0/2298146278 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00ccd0 con 0x7fdb240b1630 2024-08-15T17:58:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb15ffb700 1 -- 172.21.15.38:0/2298146278 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1c00f040 con 0x7fdb240b1630 2024-08-15T17:58:30.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb2aa19700 1 -- 172.21.15.38:0/2298146278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb18009420 con 0x7fdb240b1630 2024-08-15T17:58:30.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb15ffb700 1 -- 172.21.15.38:0/2298146278 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0146d0 con 0x7fdb240b1630 2024-08-15T17:58:30.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb2aa19700 1 -- 172.21.15.38:0/2298146278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb24144610 con 0x7fdb240b1630 2024-08-15T17:58:30.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.194+0000 7fdb15ffb700 1 -- 172.21.15.38:0/2298146278 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdb1c003020 con 0x7fdb240b1630 2024-08-15T17:58:30.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.198+0000 7fdb2aa19700 1 -- 172.21.15.38:0/2298146278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb240643a0 con 0x7fdb240b1630 2024-08-15T17:58:30.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.198+0000 7fdb15ffb700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb08060860 0x7fdb08062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:30.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.198+0000 7fdb15ffb700 1 -- 172.21.15.38:0/2298146278 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdb1c07d2a0 con 0x7fdb240b1630 2024-08-15T17:58:30.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.198+0000 7fdb29a17700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb08060860 0x7fdb08062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:30.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.198+0000 7fdb29a17700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb08060860 0x7fdb08062d10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdb200058d0 tx=0x7fdb20009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:30.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.202+0000 7fdb15ffb700 1 -- 172.21.15.38:0/2298146278 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdb1c04da40 con 0x7fdb240b1630 2024-08-15T17:58:30.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.410+0000 7fdb2aa19700 1 -- 172.21.15.38:0/2298146278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb2406f5f0 con 0x7fdb240b1630 2024-08-15T17:58:30.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.410+0000 7fdb15ffb700 1 -- 172.21.15.38:0/2298146278 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7fdb1c051060 con 0x7fdb240b1630 2024-08-15T17:58:30.413 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:30.413 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2667173752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2667173752},{"type":"v1","addr":"172.21.15.38:6835","nonce":2667173752}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-08-15T17:58:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 -- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb08060860 msgr2=0x7fdb08062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb08060860 0x7fdb08062d10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdb200058d0 tx=0x7fdb20009040 comp rx=0 tx=0).stop 2024-08-15T17:58:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 -- 172.21.15.38:0/2298146278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb240b1630 msgr2=0x7fdb2413f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb240b1630 0x7fdb2413f310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009fc0 tx=0x7fdb1c00b040 comp rx=0 tx=0).stop 2024-08-15T17:58:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 -- 172.21.15.38:0/2298146278 shutdown_connections 2024-08-15T17:58:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb08060860 0x7fdb08062d10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb240b1630 0x7fdb2413f310 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb240b0d20 0x7fdb2415c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 --2- 172.21.15.38:0/2298146278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdb2406e620 0x7fdb24171b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:30.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 -- 172.21.15.38:0/2298146278 >> 172.21.15.38:0/2298146278 conn(0x7fdb240678d0 msgr2=0x7fdb24063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:30.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 -- 172.21.15.38:0/2298146278 shutdown_connections 2024-08-15T17:58:30.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:30.414+0000 7fdb2aa19700 1 -- 172.21.15.38:0/2298146278 wait complete. 2024-08-15T17:58:30.417 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 2 2024-08-15T17:58:32.429 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:58:32.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a8a73f700 1 Processor -- start 2024-08-15T17:58:32.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a8a73f700 1 -- start start 2024-08-15T17:58:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a8a73f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a84153a40 0x7f0a840c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a8a73f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0a8415a260 0x7f0a840ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a8a73f700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a840c9e00 con 0x7f0a840b29f0 2024-08-15T17:58:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a8a73f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a840ccd20 con 0x7f0a8415a260 2024-08-15T17:58:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a8a73f700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840cc890 con 0x7f0a84153a40 2024-08-15T17:58:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a837fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0a840b29f0 0x7f0a84159b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50432/0 (socket says 172.21.15.38:50432) 2024-08-15T17:58:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a837fe700 1 -- 172.21.15.38:0/132903075 learned_addr learned my addr 172.21.15.38:0/132903075 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a83fff700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a84153a40 0x7f0a840c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a88cdc700 1 --2- 172.21.15.38:0/132903075 >> v2:172.21.15.104:3300/0 conn(0x7f0a8415a260 0x7f0a840ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014131273 0 0) 0x7f0a840c9e00 con 0x7f0a840b29f0 2024-08-15T17:58:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a64003570 con 0x7f0a840b29f0 2024-08-15T17:58:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1081661282 0 0) 0x7f0a64003570 con 0x7f0a840b29f0 2024-08-15T17:58:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a84153a40 msgr2=0x7f0a840c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a84153a40 0x7f0a840c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 >> v2:172.21.15.104:3300/0 conn(0x7f0a8415a260 msgr2=0x7f0a840ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 --2- 172.21.15.38:0/132903075 >> v2:172.21.15.104:3300/0 conn(0x7f0a8415a260 0x7f0a840ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a840cbf60 con 0x7f0a840b29f0 2024-08-15T17:58:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f0a740017f0 con 0x7f0a840b29f0 2024-08-15T17:58:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a83fff700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a84153a40 0x7f0a840c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:32.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 >> v1:172.21.15.38:6789/0 conn(0x7f0a840b29f0 legacy=0x7f0a84159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a64003a60 0x7f0a64003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a64004810 0x7f0a64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a84153a40 0x7f0a640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840ccd20 con 0x7f0a64004810 2024-08-15T17:58:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840cc890 con 0x7f0a64003a60 2024-08-15T17:58:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6400ba60 con 0x7f0a84153a40 2024-08-15T17:58:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a83fff700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a64003a60 0x7f0a64003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a837fe700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a64004810 0x7f0a64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.546+0000 7f0a88cdc700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a84153a40 0x7f0a640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a837fe700 1 -- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a84153a40 msgr2=0x7f0a640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a837fe700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a84153a40 0x7f0a640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a837fe700 1 -- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a64003a60 msgr2=0x7f0a64003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a837fe700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a64003a60 0x7f0a64003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a837fe700 1 -- 172.21.15.38:0/132903075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c009420 con 0x7f0a64004810 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a83fff700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a64003a60 0x7f0a64003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a837fe700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a64004810 0x7f0a64004c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400cc10 tx=0x7f0a74002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac09bc16b61b996 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7400e8b0 con 0x7f0a64004810 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7400ea10 con 0x7f0a64004810 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a82ffd700 1 -- 172.21.15.38:0/132903075 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7400edd0 con 0x7f0a64004810 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 -- 172.21.15.38:0/132903075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a64004810 msgr2=0x7f0a64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 --2- 172.21.15.38:0/132903075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a64004810 0x7f0a64004c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400cc10 tx=0x7f0a74002210 comp rx=0 tx=0).stop 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a88cdc700 1 -- 172.21.15.38:0/132903075 reap_dead start 2024-08-15T17:58:32.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 -- 172.21.15.38:0/132903075 shutdown_connections 2024-08-15T17:58:32.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 -- 172.21.15.38:0/132903075 >> 172.21.15.38:0/132903075 conn(0x7f0a840a9db0 msgr2=0x7f0a840ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:32.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 -- 172.21.15.38:0/132903075 shutdown_connections 2024-08-15T17:58:32.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 -- 172.21.15.38:0/132903075 wait complete. 2024-08-15T17:58:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 Processor -- start 2024-08-15T17:58:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 -- start start 2024-08-15T17:58:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a84149e90 0x7f0a84150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a84150550 0x7f0a8414b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a8414be10 0x7f0a840b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841509c0 con 0x7f0a84149e90 2024-08-15T17:58:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840acf10 con 0x7f0a84150550 2024-08-15T17:58:32.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a8a73f700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8414c600 con 0x7f0a8414be10 2024-08-15T17:58:32.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.550+0000 7f0a83fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a84149e90 0x7f0a84150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a83fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a84149e90 0x7f0a84150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48600/0 (socket says 172.21.15.38:48600) 2024-08-15T17:58:32.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a837fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a84150550 0x7f0a8414b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a83fff700 1 -- 172.21.15.38:0/1013624812 learned_addr learned my addr 172.21.15.38:0/1013624812 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:32.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a88cdc700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a8414be10 0x7f0a840b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a837fe700 1 -- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a8414be10 msgr2=0x7f0a840b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:32.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a837fe700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a8414be10 0x7f0a840b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a837fe700 1 -- 172.21.15.38:0/1013624812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a84149e90 msgr2=0x7f0a84150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:32.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a837fe700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a84149e90 0x7f0a84150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a837fe700 1 -- 172.21.15.38:0/1013624812 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8414c8e0 con 0x7f0a84150550 2024-08-15T17:58:32.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a83fff700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a84149e90 0x7f0a84150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:32.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a837fe700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a84150550 0x7f0a8414b8d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400cbe0 tx=0x7f0a7401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:32.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a7bfff700 1 -- 172.21.15.38:0/1013624812 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7400e8b0 con 0x7f0a84150550 2024-08-15T17:58:32.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a7bfff700 1 -- 172.21.15.38:0/1013624812 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a74001e10 con 0x7f0a84150550 2024-08-15T17:58:32.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a8a73f700 1 -- 172.21.15.38:0/1013624812 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a840acba0 con 0x7f0a84150550 2024-08-15T17:58:32.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a7bfff700 1 -- 172.21.15.38:0/1013624812 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74011ac0 con 0x7f0a84150550 2024-08-15T17:58:32.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a8a73f700 1 -- 172.21.15.38:0/1013624812 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8414a260 con 0x7f0a84150550 2024-08-15T17:58:32.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a8a73f700 1 -- 172.21.15.38:0/1013624812 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a840aca00 con 0x7f0a84150550 2024-08-15T17:58:32.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.554+0000 7f0a7bfff700 1 -- 172.21.15.38:0/1013624812 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0a74047b20 con 0x7f0a84150550 2024-08-15T17:58:32.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.558+0000 7f0a7bfff700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0a680608e0 0x7f0a68062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.558+0000 7f0a7bfff700 1 -- 172.21.15.38:0/1013624812 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0a74080b20 con 0x7f0a84150550 2024-08-15T17:58:32.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.558+0000 7f0a83fff700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0a680608e0 0x7f0a68062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.558+0000 7f0a83fff700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0a680608e0 0x7f0a68062d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009fd0 tx=0x7f0a6c0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:32.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.566+0000 7f0a7bfff700 1 -- 172.21.15.38:0/1013624812 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0a740512a0 con 0x7f0a84150550 2024-08-15T17:58:32.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a8a73f700 1 -- 172.21.15.38:0/1013624812 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a840aca00 con 0x7f0a84150550 2024-08-15T17:58:32.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a7bfff700 1 -- 172.21.15.38:0/1013624812 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f0a74015de0 con 0x7f0a84150550 2024-08-15T17:58:32.768 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:32.768 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2444719791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2444719791},{"type":"v1","addr":"172.21.15.38:6833","nonce":2444719791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2667173752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2667173752},{"type":"v1","addr":"172.21.15.38:6835","nonce":2667173752}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1185013072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1185013072},{"type":"v1","addr":"172.21.15.104:6835","nonce":1185013072}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3628618356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3628618356},{"type":"v1","addr":"172.21.15.104:6837","nonce":3628618356}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-08-15T17:58:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a8a73f700 1 -- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0a680608e0 msgr2=0x7f0a68062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a8a73f700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0a680608e0 0x7f0a68062d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009fd0 tx=0x7f0a6c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a8a73f700 1 -- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a84150550 msgr2=0x7f0a8414b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a8a73f700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a84150550 0x7f0a8414b8d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400cbe0 tx=0x7f0a7401a000 comp rx=0 tx=0).stop 2024-08-15T17:58:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a8a73f700 1 -- 172.21.15.38:0/1013624812 shutdown_connections 2024-08-15T17:58:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a8a73f700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0a680608e0 0x7f0a68062d90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a8a73f700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0a8414be10 0x7f0a840b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a8a73f700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a84150550 0x7f0a8414b8d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a8a73f700 1 --2- 172.21.15.38:0/1013624812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a84149e90 0x7f0a84150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.766+0000 7f0a8a73f700 1 -- 172.21.15.38:0/1013624812 >> 172.21.15.38:0/1013624812 conn(0x7f0a840a9db0 msgr2=0x7f0a840ab360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.770+0000 7f0a8a73f700 1 -- 172.21.15.38:0/1013624812 shutdown_connections 2024-08-15T17:58:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.770+0000 7f0a8a73f700 1 -- 172.21.15.38:0/1013624812 wait complete. 2024-08-15T17:58:32.772 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 3 2024-08-15T17:58:32.784 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:32.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58fa569700 1 Processor -- start 2024-08-15T17:58:32.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58fa569700 1 -- start start 2024-08-15T17:58:32.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58fa569700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f58f40ab540 0x7f58f40b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58fa569700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f406f0e0 con 0x7f58f406e370 2024-08-15T17:58:32.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58fa569700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f40aba70 con 0x7f58f406df10 2024-08-15T17:58:32.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58fa569700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f40abd50 con 0x7f58f40ab540 2024-08-15T17:58:32.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58f9d68700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f58f406e370 0x7f58f4155010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50442/0 (socket says 172.21.15.38:50442) 2024-08-15T17:58:32.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58f9d68700 1 -- 172.21.15.38:0/2484193590 learned_addr learned my addr 172.21.15.38:0/2484193590 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:32.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58f9567700 1 --2- 172.21.15.38:0/2484193590 >> v2:172.21.15.104:3300/0 conn(0x7f58f40ab540 0x7f58f40b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58e7fff700 1 -- 172.21.15.38:0/2484193590 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426002838 0 0) 0x7f58f40aba70 con 0x7f58f406df10 2024-08-15T17:58:32.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58e7fff700 1 -- 172.21.15.38:0/2484193590 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58e0003570 con 0x7f58f406df10 2024-08-15T17:58:32.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58e7fff700 1 -- 172.21.15.38:0/2484193590 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603406029 0 0) 0x7f58f406f0e0 con 0x7f58f406e370 2024-08-15T17:58:32.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58e7fff700 1 -- 172.21.15.38:0/2484193590 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58f40aba70 con 0x7f58f406e370 2024-08-15T17:58:32.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.890+0000 7f58f9567700 1 -- 172.21.15.38:0/2484193590 >> v1:172.21.15.104:6789/0 conn(0x7f58f406df10 legacy=0x7f58f406e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:32.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58f9567700 1 -- 172.21.15.38:0/2484193590 >> v1:172.21.15.38:6789/0 conn(0x7f58f406e370 legacy=0x7f58f4155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:32.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58f9567700 1 -- 172.21.15.38:0/2484193590 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f40ac030 con 0x7f58f40ab540 2024-08-15T17:58:32.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58f9567700 1 --2- 172.21.15.38:0/2484193590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f40ab540 0x7f58f40b1320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f58f00098e0 tx=0x7f58f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73b958a6cbf2d439 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:32.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58e7fff700 1 -- 172.21.15.38:0/2484193590 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0011630 con 0x7f58f40ab540 2024-08-15T17:58:32.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58e7fff700 1 -- 172.21.15.38:0/2484193590 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0011790 con 0x7f58f40ab540 2024-08-15T17:58:32.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58e7fff700 1 -- 172.21.15.38:0/2484193590 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0011b30 con 0x7f58f40ab540 2024-08-15T17:58:32.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58fa569700 1 -- 172.21.15.38:0/2484193590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f40ab540 msgr2=0x7f58f40b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:32.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58fa569700 1 --2- 172.21.15.38:0/2484193590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f40ab540 0x7f58f40b1320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f58f00098e0 tx=0x7f58f00091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58fa569700 1 -- 172.21.15.38:0/2484193590 shutdown_connections 2024-08-15T17:58:32.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58fa569700 1 --2- 172.21.15.38:0/2484193590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f40ab540 0x7f58f40b1320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58fa569700 1 -- 172.21.15.38:0/2484193590 >> 172.21.15.38:0/2484193590 conn(0x7f58f40678d0 msgr2=0x7f58f4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:32.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58fa569700 1 -- 172.21.15.38:0/2484193590 shutdown_connections 2024-08-15T17:58:32.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58fa569700 1 -- 172.21.15.38:0/2484193590 wait complete. 2024-08-15T17:58:32.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58fa569700 1 Processor -- start 2024-08-15T17:58:32.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.894+0000 7f58fa569700 1 -- start start 2024-08-15T17:58:32.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58fa569700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f406df10 0x7f58f41451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58fa569700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58f406e370 0x7f58f4148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58fa569700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40ab540 0x7f58f4146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58fa569700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f414a020 con 0x7f58f40ab540 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58fa569700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f4149410 con 0x7f58f406df10 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58fa569700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41496f0 con 0x7f58f406e370 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58f9d68700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40ab540 0x7f58f4146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58f9d68700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40ab540 0x7f58f4146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48614/0 (socket says 172.21.15.38:48614) 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58f9d68700 1 -- 172.21.15.38:0/1406876087 learned_addr learned my addr 172.21.15.38:0/1406876087 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58f9567700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f406df10 0x7f58f41451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58f8d66700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58f406e370 0x7f58f4148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58f9d68700 1 -- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58f406e370 msgr2=0x7f58f4148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58f9d68700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58f406e370 0x7f58f4148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58f9d68700 1 -- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f406df10 msgr2=0x7f58f41451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58f9d68700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f406df10 0x7f58f41451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58f9d68700 1 -- 172.21.15.38:0/1406876087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f0009420 con 0x7f58f40ab540 2024-08-15T17:58:32.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58f9d68700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40ab540 0x7f58f4146520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58ec001460 tx=0x7f58ec001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:32.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58e57fa700 1 -- 172.21.15.38:0/1406876087 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec00a940 con 0x7f58f40ab540 2024-08-15T17:58:32.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58e57fa700 1 -- 172.21.15.38:0/1406876087 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec00aaa0 con 0x7f58f40ab540 2024-08-15T17:58:32.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58fa569700 1 -- 172.21.15.38:0/1406876087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f41499d0 con 0x7f58f40ab540 2024-08-15T17:58:32.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58e57fa700 1 -- 172.21.15.38:0/1406876087 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec00ae20 con 0x7f58f40ab540 2024-08-15T17:58:32.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58fa569700 1 -- 172.21.15.38:0/1406876087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f41490a0 con 0x7f58f40ab540 2024-08-15T17:58:32.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.898+0000 7f58fa569700 1 -- 172.21.15.38:0/1406876087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f40643a0 con 0x7f58f40ab540 2024-08-15T17:58:32.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.902+0000 7f58e57fa700 1 -- 172.21.15.38:0/1406876087 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f58ec007300 con 0x7f58f40ab540 2024-08-15T17:58:32.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.902+0000 7f58e57fa700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f58d8060860 0x7f58d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:32.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.902+0000 7f58e57fa700 1 -- 172.21.15.38:0/1406876087 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f58ec07cc50 con 0x7f58f40ab540 2024-08-15T17:58:32.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.902+0000 7f58f9567700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f58d8060860 0x7f58d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:32.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.902+0000 7f58f9567700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f58d8060860 0x7f58d8062d10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f58f00093f0 tx=0x7f58f0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:32.910+0000 7f58e57fa700 1 -- 172.21.15.38:0/1406876087 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f58ec082020 con 0x7f58f40ab540 2024-08-15T17:58:33.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.074+0000 7f58fa569700 1 -- 172.21.15.38:0/1406876087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58f414a4a0 con 0x7f58f40ab540 2024-08-15T17:58:33.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.074+0000 7f58e57fa700 1 -- 172.21.15.38:0/1406876087 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12127 (secure 0 0 0) 0x7f58ec04d470 con 0x7f58f40ab540 2024-08-15T17:58:33.078 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:33.079 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:25.005981+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 -- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f58d8060860 msgr2=0x7f58d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f58d8060860 0x7f58d8062d10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f58f00093f0 tx=0x7f58f0017000 comp rx=0 tx=0).stop 2024-08-15T17:58:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 -- 172.21.15.38:0/1406876087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40ab540 msgr2=0x7f58f4146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40ab540 0x7f58f4146520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58ec001460 tx=0x7f58ec001ae0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 -- 172.21.15.38:0/1406876087 shutdown_connections 2024-08-15T17:58:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f58d8060860 0x7f58d8062d10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40ab540 0x7f58f4146520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58f406e370 0x7f58f4148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 --2- 172.21.15.38:0/1406876087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f406df10 0x7f58f41451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 -- 172.21.15.38:0/1406876087 >> 172.21.15.38:0/1406876087 conn(0x7f58f40678d0 msgr2=0x7f58f414fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:33.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 -- 172.21.15.38:0/1406876087 shutdown_connections 2024-08-15T17:58:33.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.078+0000 7f58fa569700 1 -- 172.21.15.38:0/1406876087 wait complete. 2024-08-15T17:58:33.099 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-15T17:58:33.099 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:58:33.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd260fa6700 1 Processor -- start 2024-08-15T17:58:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd260fa6700 1 -- start start 2024-08-15T17:58:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd260fa6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd25c067510 0x7fd25c0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd260fa6700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c0ce210 con 0x7fd25c1590b0 2024-08-15T17:58:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd260fa6700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c0cb440 con 0x7fd25c159480 2024-08-15T17:58:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd260fa6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c0cba00 con 0x7fd25c067510 2024-08-15T17:58:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd25b7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd25c1590b0 0x7fd25c066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50452/0 (socket says 172.21.15.38:50452) 2024-08-15T17:58:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd25b7fe700 1 -- 172.21.15.38:0/3471386570 learned_addr learned my addr 172.21.15.38:0/3471386570 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd252ffd700 1 --2- 172.21.15.38:0/3471386570 >> v2:172.21.15.104:3300/0 conn(0x7fd25c067510 0x7fd25c0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd25affd700 1 -- 172.21.15.38:0/3471386570 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275489254 0 0) 0x7fd25c0ce210 con 0x7fd25c1590b0 2024-08-15T17:58:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd25affd700 1 -- 172.21.15.38:0/3471386570 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd248003570 con 0x7fd25c1590b0 2024-08-15T17:58:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd25affd700 1 -- 172.21.15.38:0/3471386570 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940421177 0 0) 0x7fd25c0cb440 con 0x7fd25c159480 2024-08-15T17:58:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd25affd700 1 -- 172.21.15.38:0/3471386570 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd25c0ce210 con 0x7fd25c159480 2024-08-15T17:58:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd25affd700 1 -- 172.21.15.38:0/3471386570 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449155633 0 0) 0x7fd248003570 con 0x7fd25c1590b0 2024-08-15T17:58:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd252ffd700 1 -- 172.21.15.38:0/3471386570 >> v1:172.21.15.104:6789/0 conn(0x7fd25c159480 legacy=0x7fd25c0cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd252ffd700 1 -- 172.21.15.38:0/3471386570 >> v1:172.21.15.38:6789/0 conn(0x7fd25c1590b0 legacy=0x7fd25c066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.218+0000 7fd252ffd700 1 -- 172.21.15.38:0/3471386570 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0cb720 con 0x7fd25c067510 2024-08-15T17:58:33.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd252ffd700 1 --2- 172.21.15.38:0/3471386570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd25c067510 0x7fd25c0ce6a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd2440098e0 tx=0x7fd2440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce8859817956ff4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd25affd700 1 -- 172.21.15.38:0/3471386570 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c001470 con 0x7fd25c067510 2024-08-15T17:58:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd25affd700 1 -- 172.21.15.38:0/3471386570 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd244011420 con 0x7fd25c067510 2024-08-15T17:58:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd25affd700 1 -- 172.21.15.38:0/3471386570 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2440117a0 con 0x7fd25c067510 2024-08-15T17:58:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd260fa6700 1 -- 172.21.15.38:0/3471386570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd25c067510 msgr2=0x7fd25c0ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd260fa6700 1 --2- 172.21.15.38:0/3471386570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd25c067510 0x7fd25c0ce6a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd2440098e0 tx=0x7fd2440091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd260fa6700 1 -- 172.21.15.38:0/3471386570 shutdown_connections 2024-08-15T17:58:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd260fa6700 1 --2- 172.21.15.38:0/3471386570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd25c067510 0x7fd25c0ce6a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd260fa6700 1 -- 172.21.15.38:0/3471386570 >> 172.21.15.38:0/3471386570 conn(0x7fd25c0d0b90 msgr2=0x7fd25c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd260fa6700 1 -- 172.21.15.38:0/3471386570 shutdown_connections 2024-08-15T17:58:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd260fa6700 1 -- 172.21.15.38:0/3471386570 wait complete. 2024-08-15T17:58:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd260fa6700 1 Processor -- start 2024-08-15T17:58:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.222+0000 7fd260fa6700 1 -- start start 2024-08-15T17:58:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd260fa6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd25c067510 0x7fd25c15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd260fa6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd25c1590b0 0x7fd25c0c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd260fa6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c159480 0x7fd25c16d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd260fa6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0c7170 con 0x7fd25c159480 2024-08-15T17:58:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd260fa6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0c7cf0 con 0x7fd25c067510 2024-08-15T17:58:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd260fa6700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c15d4e0 con 0x7fd25c1590b0 2024-08-15T17:58:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd25bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c159480 0x7fd25c16d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd25bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c159480 0x7fd25c16d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48628/0 (socket says 172.21.15.38:48628) 2024-08-15T17:58:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd25bfff700 1 -- 172.21.15.38:0/3481280167 learned_addr learned my addr 172.21.15.38:0/3481280167 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd25b7fe700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd25c067510 0x7fd25c15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd252ffd700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd25c1590b0 0x7fd25c0c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd25bfff700 1 -- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd25c1590b0 msgr2=0x7fd25c0c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd25bfff700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd25c1590b0 0x7fd25c0c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd25bfff700 1 -- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd25c067510 msgr2=0x7fd25c15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd25bfff700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd25c067510 0x7fd25c15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd25bfff700 1 -- 172.21.15.38:0/3481280167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd244009420 con 0x7fd25c159480 2024-08-15T17:58:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd25bfff700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c159480 0x7fd25c16d250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c890 tx=0x7fd254002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd253fff700 1 -- 172.21.15.38:0/3481280167 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254002030 con 0x7fd25c159480 2024-08-15T17:58:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd253fff700 1 -- 172.21.15.38:0/3481280167 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2540021b0 con 0x7fd25c159480 2024-08-15T17:58:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd260fa6700 1 -- 172.21.15.38:0/3481280167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c0c73c0 con 0x7fd25c159480 2024-08-15T17:58:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd253fff700 1 -- 172.21.15.38:0/3481280167 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254012630 con 0x7fd25c159480 2024-08-15T17:58:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.226+0000 7fd260fa6700 1 -- 172.21.15.38:0/3481280167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c0c7980 con 0x7fd25c159480 2024-08-15T17:58:33.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.230+0000 7fd260fa6700 1 -- 172.21.15.38:0/3481280167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c0c7e80 con 0x7fd25c159480 2024-08-15T17:58:33.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.230+0000 7fd253fff700 1 -- 172.21.15.38:0/3481280167 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd254012a20 con 0x7fd25c159480 2024-08-15T17:58:33.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.230+0000 7fd253fff700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd23c060860 0x7fd23c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.230+0000 7fd25b7fe700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd23c060860 0x7fd23c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.230+0000 7fd25b7fe700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd23c060860 0x7fd23c062d10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd24c004de0 tx=0x7fd24c000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:33.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.234+0000 7fd253fff700 1 -- 172.21.15.38:0/3481280167 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd254051350 con 0x7fd25c159480 2024-08-15T17:58:33.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.242+0000 7fd253fff700 1 -- 172.21.15.38:0/3481280167 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd25404cf50 con 0x7fd25c159480 2024-08-15T17:58:33.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.414+0000 7fd260fa6700 1 -- 172.21.15.38:0/3481280167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd25c064330 con 0x7fd25c159480 2024-08-15T17:58:33.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.414+0000 7fd253fff700 1 -- 172.21.15.38:0/3481280167 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd254050570 con 0x7fd25c159480 2024-08-15T17:58:33.416 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T17:58:33.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 -- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd23c060860 msgr2=0x7fd23c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd23c060860 0x7fd23c062d10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd24c004de0 tx=0x7fd24c000bc0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 -- 172.21.15.38:0/3481280167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c159480 msgr2=0x7fd25c16d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c159480 0x7fd25c16d250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c890 tx=0x7fd254002700 comp rx=0 tx=0).stop 2024-08-15T17:58:33.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 -- 172.21.15.38:0/3481280167 shutdown_connections 2024-08-15T17:58:33.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd23c060860 0x7fd23c062d10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c159480 0x7fd25c16d250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd25c1590b0 0x7fd25c0c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 --2- 172.21.15.38:0/3481280167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd25c067510 0x7fd25c15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 -- 172.21.15.38:0/3481280167 >> 172.21.15.38:0/3481280167 conn(0x7fd25c0d0b90 msgr2=0x7fd25c0c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:33.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 -- 172.21.15.38:0/3481280167 shutdown_connections 2024-08-15T17:58:33.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.418+0000 7fd260fa6700 1 -- 172.21.15.38:0/3481280167 wait complete. 2024-08-15T17:58:33.430 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-08-15T17:58:33.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd7ec2700 1 Processor -- start 2024-08-15T17:58:33.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd7ec2700 1 -- start start 2024-08-15T17:58:33.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd7ec2700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f4dd00a9430 0x7f4dd00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd7ec2700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd00b0270 con 0x7f4dd00af030 2024-08-15T17:58:33.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd7ec2700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd00af660 con 0x7f4dd015d540 2024-08-15T17:58:33.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd7ec2700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd00af940 con 0x7f4dd00a9430 2024-08-15T17:58:33.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd545d700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4dd00af030 0x7f4dd00a8d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50456/0 (socket says 172.21.15.38:50456) 2024-08-15T17:58:33.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd545d700 1 -- 172.21.15.38:0/2752386325 learned_addr learned my addr 172.21.15.38:0/2752386325 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:33.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd645f700 1 --2- 172.21.15.38:0/2752386325 >> v2:172.21.15.104:3300/0 conn(0x7f4dd00a9430 0x7f4dd00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd4c5c700 1 -- 172.21.15.38:0/2752386325 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039681013 0 0) 0x7f4dd00b0270 con 0x7f4dd00af030 2024-08-15T17:58:33.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd4c5c700 1 -- 172.21.15.38:0/2752386325 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dbc003570 con 0x7f4dd00af030 2024-08-15T17:58:33.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd4c5c700 1 -- 172.21.15.38:0/2752386325 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491837946 0 0) 0x7f4dd00af660 con 0x7f4dd015d540 2024-08-15T17:58:33.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd4c5c700 1 -- 172.21.15.38:0/2752386325 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dd00b0270 con 0x7f4dd015d540 2024-08-15T17:58:33.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd645f700 1 -- 172.21.15.38:0/2752386325 >> v1:172.21.15.104:6789/0 conn(0x7f4dd015d540 legacy=0x7f4dd006e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd645f700 1 -- 172.21.15.38:0/2752386325 >> v1:172.21.15.38:6789/0 conn(0x7f4dd00af030 legacy=0x7f4dd00a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd4c5c700 1 -- 172.21.15.38:0/2752386325 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1650595939 0 0) 0x7f4dbc003570 con 0x7f4dd00af030 2024-08-15T17:58:33.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd645f700 1 -- 172.21.15.38:0/2752386325 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd00afc20 con 0x7f4dd00a9430 2024-08-15T17:58:33.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.534+0000 7f4dd645f700 1 --2- 172.21.15.38:0/2752386325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd00a9430 0x7f4dd00b0700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009e60 tx=0x7f4dc8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ff67f1a2085001 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:33.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd4c5c700 1 -- 172.21.15.38:0/2752386325 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800e070 con 0x7f4dd00a9430 2024-08-15T17:58:33.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd4c5c700 1 -- 172.21.15.38:0/2752386325 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc800b7a0 con 0x7f4dd00a9430 2024-08-15T17:58:33.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd4c5c700 1 -- 172.21.15.38:0/2752386325 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc001470 con 0x7f4dd00a9430 2024-08-15T17:58:33.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2752386325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd00a9430 msgr2=0x7f4dd00b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 --2- 172.21.15.38:0/2752386325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd00a9430 0x7f4dd00b0700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009e60 tx=0x7f4dc8009780 comp rx=0 tx=0).stop 2024-08-15T17:58:33.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2752386325 shutdown_connections 2024-08-15T17:58:33.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 --2- 172.21.15.38:0/2752386325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd00a9430 0x7f4dd00b0700 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2752386325 >> 172.21.15.38:0/2752386325 conn(0x7f4dd0066e00 msgr2=0x7f4dd0067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:33.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2752386325 shutdown_connections 2024-08-15T17:58:33.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2752386325 wait complete. 2024-08-15T17:58:33.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 Processor -- start 2024-08-15T17:58:33.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 -- start start 2024-08-15T17:58:33.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dd00a9430 0x7f4dd016df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dd00af030 0x7f4dd0170a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd015d540 0x7f4dd0161710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd016e450 con 0x7f4dd00af030 2024-08-15T17:58:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00c9b40 con 0x7f4dd015d540 2024-08-15T17:58:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.538+0000 7f4dd7ec2700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd0170fb0 con 0x7f4dd00a9430 2024-08-15T17:58:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd545d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dd00af030 0x7f4dd0170a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd545d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dd00af030 0x7f4dd0170a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48642/0 (socket says 172.21.15.38:48642) 2024-08-15T17:58:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd545d700 1 -- 172.21.15.38:0/2661882535 learned_addr learned my addr 172.21.15.38:0/2661882535 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd645f700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd015d540 0x7f4dd0161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd5c5e700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dd00a9430 0x7f4dd016df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd545d700 1 -- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dd00a9430 msgr2=0x7f4dd016df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd545d700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dd00a9430 0x7f4dd016df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd545d700 1 -- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd015d540 msgr2=0x7f4dd0161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd545d700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd015d540 0x7f4dd0161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd545d700 1 -- 172.21.15.38:0/2661882535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc8009a20 con 0x7f4dd00af030 2024-08-15T17:58:33.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd545d700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dd00af030 0x7f4dd0170a70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00cb90 tx=0x7f4dcc001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:33.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dc5ffb700 1 -- 172.21.15.38:0/2661882535 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc00cea0 con 0x7f4dd00af030 2024-08-15T17:58:33.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dc5ffb700 1 -- 172.21.15.38:0/2661882535 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc007e90 con 0x7f4dd00af030 2024-08-15T17:58:33.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2661882535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd00c9de0 con 0x7f4dd00af030 2024-08-15T17:58:33.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dc5ffb700 1 -- 172.21.15.38:0/2661882535 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0075d0 con 0x7f4dd00af030 2024-08-15T17:58:33.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2661882535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd01714a0 con 0x7f4dd00af030 2024-08-15T17:58:33.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.542+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2661882535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd00c9f70 con 0x7f4dd00af030 2024-08-15T17:58:33.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.546+0000 7f4dc5ffb700 1 -- 172.21.15.38:0/2661882535 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4dcc004a40 con 0x7f4dd00af030 2024-08-15T17:58:33.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.546+0000 7f4dc5ffb700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4db4060860 0x7f4db4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.546+0000 7f4dc5ffb700 1 -- 172.21.15.38:0/2661882535 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4dcc07bec0 con 0x7f4dd00af030 2024-08-15T17:58:33.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.546+0000 7f4dd5c5e700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4db4060860 0x7f4db4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.546+0000 7f4dd5c5e700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4db4060860 0x7f4db4062d10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00acc20 tx=0x7f4dc0007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:33.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.554+0000 7f4dc5ffb700 1 -- 172.21.15.38:0/2661882535 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4dcc04c6e0 con 0x7f4dd00af030 2024-08-15T17:58:33.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.722+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2661882535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4dd00645c0 con 0x7f4dd00af030 2024-08-15T17:58:33.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.722+0000 7f4dc5ffb700 1 -- 172.21.15.38:0/2661882535 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4dcc04fd00 con 0x7f4dd00af030 2024-08-15T17:58:33.725 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:33.725 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-08-15T17:58:33.725 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:58:33.725 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-08-15T17:58:33.725 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-08-15T17:58:33.725 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:58:33.726 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-08-15T17:58:33.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4db4060860 msgr2=0x7f4db4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4db4060860 0x7f4db4062d10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00acc20 tx=0x7f4dc0007d50 comp rx=0 tx=0).stop 2024-08-15T17:58:33.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2661882535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dd00af030 msgr2=0x7f4dd0170a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dd00af030 0x7f4dd0170a70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00cb90 tx=0x7f4dcc001b50 comp rx=0 tx=0).stop 2024-08-15T17:58:33.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2661882535 shutdown_connections 2024-08-15T17:58:33.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4db4060860 0x7f4db4062d10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd015d540 0x7f4dd0161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dd00af030 0x7f4dd0170a70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 --2- 172.21.15.38:0/2661882535 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4dd00a9430 0x7f4dd016df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2661882535 >> 172.21.15.38:0/2661882535 conn(0x7f4dd0066e00 msgr2=0x7f4dd0159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:33.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2661882535 shutdown_connections 2024-08-15T17:58:33.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.726+0000 7f4dd7ec2700 1 -- 172.21.15.38:0/2661882535 wait complete. 2024-08-15T17:58:33.739 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-15T17:58:33.740 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-15T17:58:33.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa19ac700 1 Processor -- start 2024-08-15T17:58:33.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa19ac700 1 -- start start 2024-08-15T17:58:33.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa19ac700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fea9c158f70 0x7fea9c0aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa19ac700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fea9c159340 0x7fea9c0686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa19ac700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea9c068be0 0x7fea9c15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa19ac700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c069040 con 0x7fea9c158f70 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa19ac700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c0af8e0 con 0x7fea9c159340 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa19ac700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c06e820 con 0x7fea9c068be0 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa09aa700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fea9c158f70 0x7fea9c0aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa09aa700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fea9c158f70 0x7fea9c0aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48648/0 (socket says 172.21.15.38:48648) 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa09aa700 1 -- 172.21.15.38:0/828277484 learned_addr learned my addr 172.21.15.38:0/828277484 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7fea9bfff700 1 --2- 172.21.15.38:0/828277484 >> v2:172.21.15.104:3300/0 conn(0x7fea9c159340 0x7fea9c0686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.854+0000 7feaa11ab700 1 --2- 172.21.15.38:0/828277484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea9c068be0 0x7fea9c15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa09aa700 1 -- 172.21.15.38:0/828277484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea9c068be0 msgr2=0x7fea9c15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa09aa700 1 --2- 172.21.15.38:0/828277484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea9c068be0 0x7fea9c15cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa09aa700 1 -- 172.21.15.38:0/828277484 >> v2:172.21.15.104:3300/0 conn(0x7fea9c159340 msgr2=0x7fea9c0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:33.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa09aa700 1 --2- 172.21.15.38:0/828277484 >> v2:172.21.15.104:3300/0 conn(0x7fea9c159340 0x7fea9c0686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa09aa700 1 -- 172.21.15.38:0/828277484 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c0af090 con 0x7fea9c158f70 2024-08-15T17:58:33.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa09aa700 1 --2- 172.21.15.38:0/828277484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea9c158f70 0x7fea9c0aeb50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0aeae0 tx=0x7fea8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac97b80e870d5deb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:33.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7fea9b7fe700 1 -- 172.21.15.38:0/828277484 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00b640 con 0x7fea9c158f70 2024-08-15T17:58:33.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7fea9b7fe700 1 -- 172.21.15.38:0/828277484 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c00b7a0 con 0x7fea9c158f70 2024-08-15T17:58:33.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7fea9b7fe700 1 -- 172.21.15.38:0/828277484 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00bb60 con 0x7fea9c158f70 2024-08-15T17:58:33.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa19ac700 1 -- 172.21.15.38:0/828277484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea9c158f70 msgr2=0x7fea9c0aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa19ac700 1 --2- 172.21.15.38:0/828277484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea9c158f70 0x7fea9c0aeb50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0aeae0 tx=0x7fea8c009250 comp rx=0 tx=0).stop 2024-08-15T17:58:33.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa19ac700 1 -- 172.21.15.38:0/828277484 shutdown_connections 2024-08-15T17:58:33.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa19ac700 1 --2- 172.21.15.38:0/828277484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea9c068be0 0x7fea9c15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa19ac700 1 --2- 172.21.15.38:0/828277484 >> v2:172.21.15.104:3300/0 conn(0x7fea9c159340 0x7fea9c0686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa19ac700 1 --2- 172.21.15.38:0/828277484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea9c158f70 0x7fea9c0aeb50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa19ac700 1 -- 172.21.15.38:0/828277484 >> 172.21.15.38:0/828277484 conn(0x7fea9c0ccfd0 msgr2=0x7fea9c06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:33.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa19ac700 1 -- 172.21.15.38:0/828277484 shutdown_connections 2024-08-15T17:58:33.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa19ac700 1 -- 172.21.15.38:0/828277484 wait complete. 2024-08-15T17:58:33.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa19ac700 1 Processor -- start 2024-08-15T17:58:33.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.858+0000 7feaa19ac700 1 -- start start 2024-08-15T17:58:33.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7feaa19ac700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea9c068be0 0x7fea9c0aae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7feaa19ac700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea9c158f70 0x7fea9c0ab510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7feaa19ac700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea9c159340 0x7fea9c0acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7feaa19ac700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0aba50 con 0x7fea9c158f70 2024-08-15T17:58:33.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7feaa19ac700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0abcb0 con 0x7fea9c068be0 2024-08-15T17:58:33.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7feaa19ac700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0abf90 con 0x7fea9c159340 2024-08-15T17:58:33.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea9bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea9c158f70 0x7fea9c0ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea9bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea9c158f70 0x7fea9c0ab510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48654/0 (socket says 172.21.15.38:48654) 2024-08-15T17:58:33.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea9bfff700 1 -- 172.21.15.38:0/4071301976 learned_addr learned my addr 172.21.15.38:0/4071301976 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:33.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7feaa11ab700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea9c159340 0x7fea9c0acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7feaa09aa700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea9c068be0 0x7fea9c0aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea9bfff700 1 -- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea9c159340 msgr2=0x7fea9c0acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea9bfff700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea9c159340 0x7fea9c0acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea9bfff700 1 -- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea9c068be0 msgr2=0x7fea9c0aae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:33.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea9bfff700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea9c068be0 0x7fea9c0aae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:33.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea9bfff700 1 -- 172.21.15.38:0/4071301976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c0094f0 con 0x7fea9c158f70 2024-08-15T17:58:33.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea9bfff700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea9c158f70 0x7fea9c0ab510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea8400ec80 tx=0x7fea840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:33.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea98ff9700 1 -- 172.21.15.38:0/4071301976 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8400c9d0 con 0x7fea9c158f70 2024-08-15T17:58:33.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea98ff9700 1 -- 172.21.15.38:0/4071301976 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8400cb30 con 0x7fea9c158f70 2024-08-15T17:58:33.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7feaa19ac700 1 -- 172.21.15.38:0/4071301976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c0ac270 con 0x7fea9c158f70 2024-08-15T17:58:33.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7feaa19ac700 1 -- 172.21.15.38:0/4071301976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c0ac830 con 0x7fea9c158f70 2024-08-15T17:58:33.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7fea98ff9700 1 -- 172.21.15.38:0/4071301976 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84013470 con 0x7fea9c158f70 2024-08-15T17:58:33.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.862+0000 7feaa19ac700 1 -- 172.21.15.38:0/4071301976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea740052b0 con 0x7fea9c158f70 2024-08-15T17:58:33.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.866+0000 7fea98ff9700 1 -- 172.21.15.38:0/4071301976 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fea84013820 con 0x7fea9c158f70 2024-08-15T17:58:33.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.866+0000 7fea98ff9700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fea80060890 0x7fea80062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:33.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.866+0000 7fea98ff9700 1 -- 172.21.15.38:0/4071301976 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fea8407cab0 con 0x7fea9c158f70 2024-08-15T17:58:33.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.866+0000 7feaa09aa700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fea80060890 0x7fea80062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:33.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.866+0000 7feaa09aa700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fea80060890 0x7fea80062d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0aeae0 tx=0x7fea8c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:33.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:33.870+0000 7fea98ff9700 1 -- 172.21.15.38:0/4071301976 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fea8404d250 con 0x7fea9c158f70 2024-08-15T17:58:34.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.034+0000 7feaa19ac700 1 -- 172.21.15.38:0/4071301976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fea74005600 con 0x7fea9c158f70 2024-08-15T17:58:34.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.246+0000 7fea98ff9700 1 -- 172.21.15.38:0/4071301976 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fea84050870 con 0x7fea9c158f70 2024-08-15T17:58:34.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.254+0000 7feaa19ac700 1 -- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fea80060890 msgr2=0x7fea80062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:34.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.254+0000 7feaa19ac700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fea80060890 0x7fea80062d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0aeae0 tx=0x7fea8c017000 comp rx=0 tx=0).stop 2024-08-15T17:58:34.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.254+0000 7feaa19ac700 1 -- 172.21.15.38:0/4071301976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea9c158f70 msgr2=0x7fea9c0ab510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:34.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.254+0000 7feaa19ac700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea9c158f70 0x7fea9c0ab510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea8400ec80 tx=0x7fea840091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:34.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.258+0000 7feaa19ac700 1 -- 172.21.15.38:0/4071301976 shutdown_connections 2024-08-15T17:58:34.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.258+0000 7feaa19ac700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fea80060890 0x7fea80062d40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:34.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.258+0000 7feaa19ac700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea9c159340 0x7fea9c0acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:34.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.258+0000 7feaa19ac700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea9c158f70 0x7fea9c0ab510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:34.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.258+0000 7feaa19ac700 1 --2- 172.21.15.38:0/4071301976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea9c068be0 0x7fea9c0aae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:34.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.258+0000 7feaa19ac700 1 -- 172.21.15.38:0/4071301976 >> 172.21.15.38:0/4071301976 conn(0x7fea9c0ccfd0 msgr2=0x7fea9c0afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:34.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.258+0000 7feaa19ac700 1 -- 172.21.15.38:0/4071301976 shutdown_connections 2024-08-15T17:58:34.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.258+0000 7feaa19ac700 1 -- 172.21.15.38:0/4071301976 wait complete. 2024-08-15T17:58:34.260 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-08-15T17:58:34.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-15T17:58:34.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.390+0000 7f06ce02a700 1 Processor -- start 2024-08-15T17:58:34.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.390+0000 7f06ce02a700 1 -- start start 2024-08-15T17:58:34.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06ce02a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f06c8068ca0 0x7f06c80b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06ce02a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80b2390 0x7f06c80c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06ce02a700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c80ad910 con 0x7f06c80688d0 2024-08-15T17:58:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06ce02a700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c80adbf0 con 0x7f06c8068ca0 2024-08-15T17:58:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06ce02a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c80aded0 con 0x7f06c80b2390 2024-08-15T17:58:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06cd829700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80b2390 0x7f06c80c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06cc827700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f06c8068ca0 0x7f06c80b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06cd829700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80b2390 0x7f06c80c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:35262/0 (socket says 172.21.15.38:35262) 2024-08-15T17:58:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06cd829700 1 -- 172.21.15.38:0/783196909 learned_addr learned my addr 172.21.15.38:0/783196909 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06cd028700 1 --1- 172.21.15.38:0/783196909 >> v1:172.21.15.104:6789/0 conn(0x7f06c80688d0 0x7f06c80ae9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:56930/0 (socket says 172.21.15.38:56930) 2024-08-15T17:58:34.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06cc827700 1 -- 172.21.15.38:0/783196909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80b2390 msgr2=0x7f06c80c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:34.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06cc827700 1 --2- 172.21.15.38:0/783196909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80b2390 0x7f06c80c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:34.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06cc827700 1 -- 172.21.15.38:0/783196909 >> v1:172.21.15.104:6789/0 conn(0x7f06c80688d0 legacy=0x7f06c80ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:34.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06cc827700 1 -- 172.21.15.38:0/783196909 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c80ae1b0 con 0x7f06c8068ca0 2024-08-15T17:58:34.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06cd829700 1 --2- 172.21.15.38:0/783196909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80b2390 0x7f06c80c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:34.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06cc827700 1 --2- 172.21.15.38:0/783196909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06c8068ca0 0x7f06c80b1e50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009910 tx=0x7f06b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551ec955dbee92c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:34.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06bffff700 1 -- 172.21.15.38:0/783196909 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b800b640 con 0x7f06c8068ca0 2024-08-15T17:58:34.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06bffff700 1 -- 172.21.15.38:0/783196909 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b800b7a0 con 0x7f06c8068ca0 2024-08-15T17:58:34.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06bffff700 1 -- 172.21.15.38:0/783196909 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b800bb60 con 0x7f06c8068ca0 2024-08-15T17:58:34.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06ce02a700 1 -- 172.21.15.38:0/783196909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06c8068ca0 msgr2=0x7f06c80b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:34.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.394+0000 7f06ce02a700 1 --2- 172.21.15.38:0/783196909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06c8068ca0 0x7f06c80b1e50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009910 tx=0x7f06b80091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.398+0000 7f06ce02a700 1 -- 172.21.15.38:0/783196909 shutdown_connections 2024-08-15T17:58:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.398+0000 7f06ce02a700 1 --2- 172.21.15.38:0/783196909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80b2390 0x7f06c80c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.398+0000 7f06ce02a700 1 --2- 172.21.15.38:0/783196909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06c8068ca0 0x7f06c80b1e50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.398+0000 7f06ce02a700 1 -- 172.21.15.38:0/783196909 >> 172.21.15.38:0/783196909 conn(0x7f06c806d8d0 msgr2=0x7f06c806dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.398+0000 7f06ce02a700 1 -- 172.21.15.38:0/783196909 shutdown_connections 2024-08-15T17:58:34.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.398+0000 7f06ce02a700 1 -- 172.21.15.38:0/783196909 wait complete. 2024-08-15T17:58:34.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.398+0000 7f06ce02a700 1 Processor -- start 2024-08-15T17:58:34.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.398+0000 7f06ce02a700 1 -- start start 2024-08-15T17:58:34.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06ce02a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80688d0 0x7f06c80c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:34.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06ce02a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06c8068ca0 0x7f06c8156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06ce02a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c80b2390 0x7f06c8137930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06ce02a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c8156de0 con 0x7f06c80b2390 2024-08-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06ce02a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c81570c0 con 0x7f06c8068ca0 2024-08-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06ce02a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c81573a0 con 0x7f06c80688d0 2024-08-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06cd829700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c80b2390 0x7f06c8137930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06cd829700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c80b2390 0x7f06c8137930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48664/0 (socket says 172.21.15.38:48664) 2024-08-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06cd829700 1 -- 172.21.15.38:0/4066998820 learned_addr learned my addr 172.21.15.38:0/4066998820 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06cc827700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06c8068ca0 0x7f06c8156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06cd028700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80688d0 0x7f06c80c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06cd829700 1 -- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80688d0 msgr2=0x7f06c80c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06cd829700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80688d0 0x7f06c80c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06cd829700 1 -- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06c8068ca0 msgr2=0x7f06c8156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06cd829700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06c8068ca0 0x7f06c8156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06cd829700 1 -- 172.21.15.38:0/4066998820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0009ae0 con 0x7f06c80b2390 2024-08-15T17:58:34.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06cd829700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c80b2390 0x7f06c8137930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009f30 tx=0x7f06c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:34.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06bd7fa700 1 -- 172.21.15.38:0/4066998820 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0018070 con 0x7f06c80b2390 2024-08-15T17:58:34.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06bd7fa700 1 -- 172.21.15.38:0/4066998820 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c000f040 con 0x7f06c80b2390 2024-08-15T17:58:34.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06ce02a700 1 -- 172.21.15.38:0/4066998820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b8009420 con 0x7f06c80b2390 2024-08-15T17:58:34.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06bd7fa700 1 -- 172.21.15.38:0/4066998820 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c000cc00 con 0x7f06c80b2390 2024-08-15T17:58:34.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06ce02a700 1 -- 172.21.15.38:0/4066998820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c8157960 con 0x7f06c80b2390 2024-08-15T17:58:34.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06ce02a700 1 -- 172.21.15.38:0/4066998820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c81574e0 con 0x7f06c80b2390 2024-08-15T17:58:34.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.402+0000 7f06bd7fa700 1 -- 172.21.15.38:0/4066998820 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f06c0003020 con 0x7f06c80b2390 2024-08-15T17:58:34.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.406+0000 7f06bd7fa700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06ac060860 0x7f06ac062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:34.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.406+0000 7f06bd7fa700 1 -- 172.21.15.38:0/4066998820 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f06c007cde0 con 0x7f06c80b2390 2024-08-15T17:58:34.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.406+0000 7f06cd028700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06ac060860 0x7f06ac062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:34.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.406+0000 7f06cd028700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06ac060860 0x7f06ac062d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f06c4007fd0 tx=0x7f06c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:34.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.410+0000 7f06bd7fa700 1 -- 172.21.15.38:0/4066998820 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f06c81574e0 con 0x7f06c80b2390 2024-08-15T17:58:34.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:34.574+0000 7f06ce02a700 1 -- 172.21.15.38:0/4066998820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f06c81574e0 con 0x7f06c80b2390 2024-08-15T17:58:35.131 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:58:35.134 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:58:35.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.250+0000 7f06bd7fa700 1 -- 172.21.15.38:0/4066998820 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f06c81574e0 con 0x7f06c80b2390 2024-08-15T17:58:35.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 -- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06ac060860 msgr2=0x7f06ac062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:35.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06ac060860 0x7f06ac062d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f06c4007fd0 tx=0x7f06c4009040 comp rx=0 tx=0).stop 2024-08-15T17:58:35.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 -- 172.21.15.38:0/4066998820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c80b2390 msgr2=0x7f06c8137930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:35.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c80b2390 0x7f06c8137930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009f30 tx=0x7f06c000b040 comp rx=0 tx=0).stop 2024-08-15T17:58:35.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 -- 172.21.15.38:0/4066998820 shutdown_connections 2024-08-15T17:58:35.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06ac060860 0x7f06ac062d10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:35.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c80b2390 0x7f06c8137930 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:35.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06c8068ca0 0x7f06c8156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:35.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 --2- 172.21.15.38:0/4066998820 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06c80688d0 0x7f06c80c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:35.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 -- 172.21.15.38:0/4066998820 >> 172.21.15.38:0/4066998820 conn(0x7f06c806d8d0 msgr2=0x7f06c80c86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:35.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 -- 172.21.15.38:0/4066998820 shutdown_connections 2024-08-15T17:58:35.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.262+0000 7f06ce02a700 1 -- 172.21.15.38:0/4066998820 wait complete. 2024-08-15T17:58:35.271 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-08-15T17:58:35.284 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-15T17:58:35.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.394+0000 7f2c896b0700 1 Processor -- start 2024-08-15T17:58:35.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.394+0000 7f2c896b0700 1 -- start start 2024-08-15T17:58:35.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.394+0000 7f2c896b0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2c8415a500 0x7f2c840ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:35.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.394+0000 7f2c896b0700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2c8415a8d0 0x7f2c840686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:35.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c896b0700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c840b1540 con 0x7f2c840b2250 2024-08-15T17:58:35.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c896b0700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8406ed10 con 0x7f2c8415a500 2024-08-15T17:58:35.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c896b0700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c840b1dc0 con 0x7f2c8415a8d0 2024-08-15T17:58:35.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c82ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2c8415a500 0x7f2c840ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:35.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c82ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2c8415a500 0x7f2c840ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48676/0 (socket says 172.21.15.38:48676) 2024-08-15T17:58:35.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c82ffd700 1 -- 172.21.15.38:0/2358920784 learned_addr learned my addr 172.21.15.38:0/2358920784 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:35.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c827fc700 1 --2- 172.21.15.38:0/2358920784 >> v2:172.21.15.104:3300/0 conn(0x7f2c8415a8d0 0x7f2c840686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:35.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c82ffd700 1 -- 172.21.15.38:0/2358920784 >> v2:172.21.15.104:3300/0 conn(0x7f2c8415a8d0 msgr2=0x7f2c840686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:35.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c82ffd700 1 --2- 172.21.15.38:0/2358920784 >> v2:172.21.15.104:3300/0 conn(0x7f2c8415a8d0 0x7f2c840686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c82ffd700 1 -- 172.21.15.38:0/2358920784 >> v1:172.21.15.38:6789/0 conn(0x7f2c840b2250 legacy=0x7f2c840699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c81ffb700 1 -- 172.21.15.38:0/2358920784 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037884173 0 0) 0x7f2c840b1540 con 0x7f2c840b2250 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c82ffd700 1 -- 172.21.15.38:0/2358920784 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8406ea30 con 0x7f2c8415a500 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c82ffd700 1 --2- 172.21.15.38:0/2358920784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c8415a500 0x7f2c840ae830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c8406d6e0 tx=0x7f2c74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22482f96593829d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c81ffb700 1 -- 172.21.15.38:0/2358920784 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400b860 con 0x7f2c8415a500 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c81ffb700 1 -- 172.21.15.38:0/2358920784 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7400b9c0 con 0x7f2c8415a500 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c81ffb700 1 -- 172.21.15.38:0/2358920784 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400bd60 con 0x7f2c8415a500 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c896b0700 1 -- 172.21.15.38:0/2358920784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c8415a500 msgr2=0x7f2c840ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c896b0700 1 --2- 172.21.15.38:0/2358920784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c8415a500 0x7f2c840ae830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c8406d6e0 tx=0x7f2c74009250 comp rx=0 tx=0).stop 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c896b0700 1 -- 172.21.15.38:0/2358920784 shutdown_connections 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c896b0700 1 --2- 172.21.15.38:0/2358920784 >> v2:172.21.15.104:3300/0 conn(0x7f2c8415a8d0 0x7f2c840686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c896b0700 1 --2- 172.21.15.38:0/2358920784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c8415a500 0x7f2c840ae830 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:35.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c896b0700 1 -- 172.21.15.38:0/2358920784 >> 172.21.15.38:0/2358920784 conn(0x7f2c840c9bf0 msgr2=0x7f2c8406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:35.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c896b0700 1 -- 172.21.15.38:0/2358920784 shutdown_connections 2024-08-15T17:58:35.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.398+0000 7f2c896b0700 1 -- 172.21.15.38:0/2358920784 wait complete. 2024-08-15T17:58:35.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c896b0700 1 Processor -- start 2024-08-15T17:58:35.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c896b0700 1 -- start start 2024-08-15T17:58:35.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c896b0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c840b2250 0x7f2c8414dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:35.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c896b0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c8415a500 0x7f2c8416a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:35.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c896b0700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c8415a8d0 0x7f2c841675e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:35.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c896b0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8414e150 con 0x7f2c8415a500 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c896b0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c84151ee0 con 0x7f2c840b2250 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c896b0700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c841521c0 con 0x7f2c8415a8d0 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c827fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c8415a500 0x7f2c8416a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c827fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c8415a500 0x7f2c8416a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48680/0 (socket says 172.21.15.38:48680) 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c827fc700 1 -- 172.21.15.38:0/962944276 learned_addr learned my addr 172.21.15.38:0/962944276 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c82ffd700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c840b2250 0x7f2c8414dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c837fe700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c8415a8d0 0x7f2c841675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c827fc700 1 -- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c8415a8d0 msgr2=0x7f2c841675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c827fc700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c8415a8d0 0x7f2c841675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c827fc700 1 -- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c840b2250 msgr2=0x7f2c8414dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c827fc700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c840b2250 0x7f2c8414dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c827fc700 1 -- 172.21.15.38:0/962944276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740094f0 con 0x7f2c8415a500 2024-08-15T17:58:35.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c827fc700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c8415a500 0x7f2c8416a310 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800d860 tx=0x7f2c78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:35.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c6affd700 1 -- 172.21.15.38:0/962944276 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c780125f0 con 0x7f2c8415a500 2024-08-15T17:58:35.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c6affd700 1 -- 172.21.15.38:0/962944276 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c78012750 con 0x7f2c8415a500 2024-08-15T17:58:35.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c896b0700 1 -- 172.21.15.38:0/962944276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8414eae0 con 0x7f2c8415a500 2024-08-15T17:58:35.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c6affd700 1 -- 172.21.15.38:0/962944276 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78012b10 con 0x7f2c8415a500 2024-08-15T17:58:35.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.402+0000 7f2c896b0700 1 -- 172.21.15.38:0/962944276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8414f0a0 con 0x7f2c8415a500 2024-08-15T17:58:35.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.406+0000 7f2c896b0700 1 -- 172.21.15.38:0/962944276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c840645c0 con 0x7f2c8415a500 2024-08-15T17:58:35.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.406+0000 7f2c6affd700 1 -- 172.21.15.38:0/962944276 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2c7801e460 con 0x7f2c8415a500 2024-08-15T17:58:35.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.406+0000 7f2c6affd700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2c70060890 0x7f2c70062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:35.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.406+0000 7f2c6affd700 1 -- 172.21.15.38:0/962944276 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f2c7807d0c0 con 0x7f2c8415a500 2024-08-15T17:58:35.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.406+0000 7f2c82ffd700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2c70060890 0x7f2c70062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:35.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.406+0000 7f2c82ffd700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2c70060890 0x7f2c70062d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c8406d6e0 tx=0x7f2c74017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:35.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.414+0000 7f2c6affd700 1 -- 172.21.15.38:0/962944276 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2c7804d570 con 0x7f2c8415a500 2024-08-15T17:58:35.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:35.614+0000 7f2c896b0700 1 -- 172.21.15.38:0/962944276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2c840ca440 con 0x7f2c8415a500 2024-08-15T17:58:36.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.282+0000 7f2c6affd700 1 -- 172.21.15.38:0/962944276 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2c78050b90 con 0x7f2c8415a500 2024-08-15T17:58:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 -- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2c70060890 msgr2=0x7f2c70062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2c70060890 0x7f2c70062d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c8406d6e0 tx=0x7f2c74017000 comp rx=0 tx=0).stop 2024-08-15T17:58:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 -- 172.21.15.38:0/962944276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c8415a500 msgr2=0x7f2c8416a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c8415a500 0x7f2c8416a310 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800d860 tx=0x7f2c78008040 comp rx=0 tx=0).stop 2024-08-15T17:58:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 -- 172.21.15.38:0/962944276 shutdown_connections 2024-08-15T17:58:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2c70060890 0x7f2c70062d40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c8415a8d0 0x7f2c841675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c8415a500 0x7f2c8416a310 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 --2- 172.21.15.38:0/962944276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c840b2250 0x7f2c8414dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 -- 172.21.15.38:0/962944276 >> 172.21.15.38:0/962944276 conn(0x7f2c840c9bf0 msgr2=0x7f2c84156150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:36.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 -- 172.21.15.38:0/962944276 shutdown_connections 2024-08-15T17:58:36.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.290+0000 7f2c896b0700 1 -- 172.21.15.38:0/962944276 wait complete. 2024-08-15T17:58:36.294 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 2 and data pool 3 2024-08-15T17:58:36.308 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:36.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4514596700 1 Processor -- start 2024-08-15T17:58:36.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4514596700 1 -- start start 2024-08-15T17:58:36.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4514596700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c156f80 0x7f450c0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4514596700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450c0adf50 con 0x7f450c06f040 2024-08-15T17:58:36.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4514596700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450c0ae920 con 0x7f450c0ace80 2024-08-15T17:58:36.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4514596700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0ad340 con 0x7f450c156f80 2024-08-15T17:58:36.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4512b33700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f450c06f040 0x7f450c0aab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50480/0 (socket says 172.21.15.38:50480) 2024-08-15T17:58:36.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4512b33700 1 -- 172.21.15.38:0/824367341 learned_addr learned my addr 172.21.15.38:0/824367341 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:36.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4512332700 1 --2- 172.21.15.38:0/824367341 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c156f80 0x7f450c0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4511330700 1 -- 172.21.15.38:0/824367341 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462665513 0 0) 0x7f450c0adf50 con 0x7f450c06f040 2024-08-15T17:58:36.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4511330700 1 -- 172.21.15.38:0/824367341 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44f4003570 con 0x7f450c06f040 2024-08-15T17:58:36.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4511330700 1 -- 172.21.15.38:0/824367341 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898726015 0 0) 0x7f450c0ae920 con 0x7f450c0ace80 2024-08-15T17:58:36.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4511330700 1 -- 172.21.15.38:0/824367341 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f450c0adf50 con 0x7f450c0ace80 2024-08-15T17:58:36.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4512332700 1 -- 172.21.15.38:0/824367341 >> v1:172.21.15.104:6789/0 conn(0x7f450c0ace80 legacy=0x7f450c157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4512332700 1 -- 172.21.15.38:0/824367341 >> v1:172.21.15.38:6789/0 conn(0x7f450c06f040 legacy=0x7f450c0aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.406+0000 7f4512332700 1 -- 172.21.15.38:0/824367341 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c0ad620 con 0x7f450c156f80 2024-08-15T17:58:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.410+0000 7f4512332700 1 --2- 172.21.15.38:0/824367341 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c156f80 0x7f450c0ae3e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4504009e90 tx=0x7f4504009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e319c96fd7ae60c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.410+0000 7f4511330700 1 -- 172.21.15.38:0/824367341 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450400e070 con 0x7f450c156f80 2024-08-15T17:58:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.410+0000 7f4511330700 1 -- 172.21.15.38:0/824367341 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450400bb80 con 0x7f450c156f80 2024-08-15T17:58:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.410+0000 7f4511330700 1 -- 172.21.15.38:0/824367341 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504012430 con 0x7f450c156f80 2024-08-15T17:58:36.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.410+0000 7f4514596700 1 -- 172.21.15.38:0/824367341 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c156f80 msgr2=0x7f450c0ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.410+0000 7f4514596700 1 --2- 172.21.15.38:0/824367341 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c156f80 0x7f450c0ae3e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4504009e90 tx=0x7f4504009780 comp rx=0 tx=0).stop 2024-08-15T17:58:36.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.410+0000 7f4514596700 1 -- 172.21.15.38:0/824367341 shutdown_connections 2024-08-15T17:58:36.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.410+0000 7f4514596700 1 --2- 172.21.15.38:0/824367341 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c156f80 0x7f450c0ae3e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.410+0000 7f4514596700 1 -- 172.21.15.38:0/824367341 >> 172.21.15.38:0/824367341 conn(0x7f450c066e00 msgr2=0x7f450c067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:36.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.410+0000 7f4514596700 1 -- 172.21.15.38:0/824367341 shutdown_connections 2024-08-15T17:58:36.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.410+0000 7f4514596700 1 -- 172.21.15.38:0/824367341 wait complete. 2024-08-15T17:58:36.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4514596700 1 Processor -- start 2024-08-15T17:58:36.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4514596700 1 -- start start 2024-08-15T17:58:36.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4514596700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c06f040 0x7f450c0c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4514596700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c0ace80 0x7f450c1692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4514596700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f450c156f80 0x7f450c151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4514596700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0c55f0 con 0x7f450c06f040 2024-08-15T17:58:36.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4514596700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c16cdd0 con 0x7f450c156f80 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4514596700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c16d0b0 con 0x7f450c0ace80 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4512332700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c06f040 0x7f450c0c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4512332700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c06f040 0x7f450c0c50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48684/0 (socket says 172.21.15.38:48684) 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4512332700 1 -- 172.21.15.38:0/3752329547 learned_addr learned my addr 172.21.15.38:0/3752329547 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4511b31700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c0ace80 0x7f450c1692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4512b33700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f450c156f80 0x7f450c151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4512332700 1 -- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c0ace80 msgr2=0x7f450c1692b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4512332700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c0ace80 0x7f450c1692b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4512332700 1 -- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f450c156f80 msgr2=0x7f450c151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4512332700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f450c156f80 0x7f450c151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4512332700 1 -- 172.21.15.38:0/3752329547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4504009a20 con 0x7f450c06f040 2024-08-15T17:58:36.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.414+0000 7f4511b31700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c0ace80 0x7f450c1692b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:36.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.418+0000 7f4512332700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c06f040 0x7f450c0c50b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f45040097c0 tx=0x7f4504019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:36.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.418+0000 7f450a7fc700 1 -- 172.21.15.38:0/3752329547 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504009420 con 0x7f450c06f040 2024-08-15T17:58:36.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.418+0000 7f4514596700 1 -- 172.21.15.38:0/3752329547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c16d390 con 0x7f450c06f040 2024-08-15T17:58:36.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.418+0000 7f450a7fc700 1 -- 172.21.15.38:0/3752329547 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450400be40 con 0x7f450c06f040 2024-08-15T17:58:36.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.418+0000 7f450a7fc700 1 -- 172.21.15.38:0/3752329547 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504026620 con 0x7f450c06f040 2024-08-15T17:58:36.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.418+0000 7f4514596700 1 -- 172.21.15.38:0/3752329547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450c0979c0 con 0x7f450c06f040 2024-08-15T17:58:36.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.418+0000 7f4514596700 1 -- 172.21.15.38:0/3752329547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450c097bb0 con 0x7f450c06f040 2024-08-15T17:58:36.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.418+0000 7f450a7fc700 1 -- 172.21.15.38:0/3752329547 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f450402f450 con 0x7f450c06f040 2024-08-15T17:58:36.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.418+0000 7f450a7fc700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44f8060890 0x7f44f8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.418+0000 7f450a7fc700 1 -- 172.21.15.38:0/3752329547 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f450407e640 con 0x7f450c06f040 2024-08-15T17:58:36.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.422+0000 7f4511b31700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44f8060890 0x7f44f8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.422+0000 7f4511b31700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44f8060890 0x7f44f8062d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f450c158040 tx=0x7f4500007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.426+0000 7f450a7fc700 1 -- 172.21.15.38:0/3752329547 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f450404e9f0 con 0x7f450c06f040 2024-08-15T17:58:36.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.590+0000 7f4514596700 1 -- 172.21.15.38:0/3752329547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f450c0645c0 con 0x7f450c06f040 2024-08-15T17:58:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f450a7fc700 1 -- 172.21.15.38:0/3752329547 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f4504052010 con 0x7f450c06f040 2024-08-15T17:58:36.595 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:36.595 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":20,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:35.621535+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 -- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44f8060890 msgr2=0x7f44f8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44f8060890 0x7f44f8062d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f450c158040 tx=0x7f4500007d30 comp rx=0 tx=0).stop 2024-08-15T17:58:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 -- 172.21.15.38:0/3752329547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c06f040 msgr2=0x7f450c0c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c06f040 0x7f450c0c50b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f45040097c0 tx=0x7f4504019000 comp rx=0 tx=0).stop 2024-08-15T17:58:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 -- 172.21.15.38:0/3752329547 shutdown_connections 2024-08-15T17:58:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44f8060890 0x7f44f8062d40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f450c156f80 0x7f450c151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f450c0ace80 0x7f450c1692b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 --2- 172.21.15.38:0/3752329547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c06f040 0x7f450c0c50b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 -- 172.21.15.38:0/3752329547 >> 172.21.15.38:0/3752329547 conn(0x7f450c066e00 msgr2=0x7f450c1581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:36.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 -- 172.21.15.38:0/3752329547 shutdown_connections 2024-08-15T17:58:36.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.594+0000 7f4514596700 1 -- 172.21.15.38:0/3752329547 wait complete. 2024-08-15T17:58:36.610 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:36.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa1ad95700 1 Processor -- start 2024-08-15T17:58:36.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa1ad95700 1 -- start start 2024-08-15T17:58:36.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa1ad95700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa140b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa1ad95700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1415eae0 con 0x7ffa14068810 2024-08-15T17:58:36.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa1ad95700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa14069e00 con 0x7ffa140ae950 2024-08-15T17:58:36.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa1ad95700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1415df60 con 0x7ffa14157ad0 2024-08-15T17:58:36.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa19592700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ffa14068810 0x7ffa14068fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40036/0 (socket says 172.21.15.38:40036) 2024-08-15T17:58:36.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa19592700 1 -- 172.21.15.38:0/2825833557 learned_addr learned my addr 172.21.15.38:0/2825833557 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:36.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa19d93700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa140b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655217871 0 0) 0x7ffa1415eae0 con 0x7ffa14068810 2024-08-15T17:58:36.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa00003570 con 0x7ffa14068810 2024-08-15T17:58:36.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914209048 0 0) 0x7ffa14069e00 con 0x7ffa140ae950 2024-08-15T17:58:36.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa1415eae0 con 0x7ffa140ae950 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3798483587 0 0) 0x7ffa00003570 con 0x7ffa14068810 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 msgr2=0x7ffa140b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa140b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 >> v1:172.21.15.104:6789/0 conn(0x7ffa140ae950 legacy=0x7ffa140696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1415e240 con 0x7ffa14068810 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7ffa0c0017f0 con 0x7ffa14068810 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 >> v1:172.21.15.38:6789/0 conn(0x7ffa14068810 legacy=0x7ffa14068fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00003c20 0x7ffa00004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa00004d50 0x7ffa000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1415df60 con 0x7ffa00003c20 2024-08-15T17:58:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0000bbf0 con 0x7ffa00004d50 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0000bdb0 con 0x7ffa14157ad0 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa19d93700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00003c20 0x7ffa00004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa19592700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.718+0000 7ffa1a594700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa00004d50 0x7ffa000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa19592700 1 -- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa00004d50 msgr2=0x7ffa000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa19592700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa00004d50 0x7ffa000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa19592700 1 -- 172.21.15.38:0/2825833557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00003c20 msgr2=0x7ffa00004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa19592700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00003c20 0x7ffa00004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa19592700 1 -- 172.21.15.38:0/2825833557 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa00003570 con 0x7ffa14157ad0 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa19d93700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa00003c20 0x7ffa00004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa19592700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa00004810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00a8f0 tx=0x7ffa0c002190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23e233db1fecb669 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0070d0 con 0x7ffa14157ad0 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c007250 con 0x7ffa14157ad0 2024-08-15T17:58:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa18d91700 1 -- 172.21.15.38:0/2825833557 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c007610 con 0x7ffa14157ad0 2024-08-15T17:58:36.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa1ad95700 1 -- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 msgr2=0x7ffa00004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa1ad95700 1 --2- 172.21.15.38:0/2825833557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa00004810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00a8f0 tx=0x7ffa0c002190 comp rx=0 tx=0).stop 2024-08-15T17:58:36.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa1a594700 1 -- 172.21.15.38:0/2825833557 reap_dead start 2024-08-15T17:58:36.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa1ad95700 1 -- 172.21.15.38:0/2825833557 shutdown_connections 2024-08-15T17:58:36.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa1ad95700 1 -- 172.21.15.38:0/2825833557 >> 172.21.15.38:0/2825833557 conn(0x7ffa140ce990 msgr2=0x7ffa1406c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:36.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa1ad95700 1 -- 172.21.15.38:0/2825833557 shutdown_connections 2024-08-15T17:58:36.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.722+0000 7ffa1ad95700 1 -- 172.21.15.38:0/2825833557 wait complete. 2024-08-15T17:58:36.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa1ad95700 1 Processor -- start 2024-08-15T17:58:36.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa1ad95700 1 -- start start 2024-08-15T17:58:36.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa1ad95700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa141542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa1ad95700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa140cd260 0x7ffa14160f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa1ad95700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa140cd6d0 0x7ffa14161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa1ad95700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141547f0 con 0x7ffa140cd260 2024-08-15T17:58:36.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa1ad95700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1415c980 con 0x7ffa140cd6d0 2024-08-15T17:58:36.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa1ad95700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1415be00 con 0x7ffa14157ad0 2024-08-15T17:58:36.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa19592700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa140cd260 0x7ffa14160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa19592700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa140cd260 0x7ffa14160f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53482/0 (socket says 172.21.15.38:53482) 2024-08-15T17:58:36.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa19592700 1 -- 172.21.15.38:0/4235786864 learned_addr learned my addr 172.21.15.38:0/4235786864 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:36.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa19d93700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa141542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa1a594700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa140cd6d0 0x7ffa14161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa19592700 1 -- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 msgr2=0x7ffa141542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa19592700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa141542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa19592700 1 -- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa140cd6d0 msgr2=0x7ffa14161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa19592700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa140cd6d0 0x7ffa14161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa19592700 1 -- 172.21.15.38:0/4235786864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1415c0e0 con 0x7ffa140cd260 2024-08-15T17:58:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa19d93700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa141542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa19592700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa140cd260 0x7ffa14160f10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00aee0 tx=0x7ffa0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa09ffb700 1 -- 172.21.15.38:0/4235786864 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0075f0 con 0x7ffa140cd260 2024-08-15T17:58:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa09ffb700 1 -- 172.21.15.38:0/4235786864 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c0070d0 con 0x7ffa140cd260 2024-08-15T17:58:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa1ad95700 1 -- 172.21.15.38:0/4235786864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1415c3c0 con 0x7ffa140cd260 2024-08-15T17:58:36.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa09ffb700 1 -- 172.21.15.38:0/4235786864 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c007980 con 0x7ffa140cd260 2024-08-15T17:58:36.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.726+0000 7ffa1ad95700 1 -- 172.21.15.38:0/4235786864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1415b9e0 con 0x7ffa140cd260 2024-08-15T17:58:36.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.730+0000 7ffa1ad95700 1 -- 172.21.15.38:0/4235786864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1415f780 con 0x7ffa140cd260 2024-08-15T17:58:36.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.730+0000 7ffa09ffb700 1 -- 172.21.15.38:0/4235786864 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ffa0c0017b0 con 0x7ffa140cd260 2024-08-15T17:58:36.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.730+0000 7ffa09ffb700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9f8060860 0x7ff9f8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:36.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.730+0000 7ffa09ffb700 1 -- 172.21.15.38:0/4235786864 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffa0c07ec20 con 0x7ffa140cd260 2024-08-15T17:58:36.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.730+0000 7ffa19d93700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9f8060860 0x7ff9f8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:36.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.730+0000 7ffa19d93700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9f8060860 0x7ff9f8062d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa04006f00 tx=0x7ffa040091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:36.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.738+0000 7ffa09ffb700 1 -- 172.21.15.38:0/4235786864 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ffa1415f780 con 0x7ffa140cd260 2024-08-15T17:58:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.902+0000 7ffa1ad95700 1 -- 172.21.15.38:0/4235786864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa1415f780 con 0x7ffa140cd260 2024-08-15T17:58:36.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.902+0000 7ffa09ffb700 1 -- 172.21.15.38:0/4235786864 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15363 (secure 0 0 0) 0x7ffa1415f780 con 0x7ffa140cd260 2024-08-15T17:58:36.905 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:36.905 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":20,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:35.621535+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:36.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 -- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9f8060860 msgr2=0x7ff9f8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9f8060860 0x7ff9f8062d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa04006f00 tx=0x7ffa040091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 -- 172.21.15.38:0/4235786864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa140cd260 msgr2=0x7ffa14160f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:36.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa140cd260 0x7ffa14160f10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00aee0 tx=0x7ffa0c016000 comp rx=0 tx=0).stop 2024-08-15T17:58:36.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 -- 172.21.15.38:0/4235786864 shutdown_connections 2024-08-15T17:58:36.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9f8060860 0x7ff9f8062d10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa140cd6d0 0x7ffa14161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa140cd260 0x7ffa14160f10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 --2- 172.21.15.38:0/4235786864 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa14157ad0 0x7ffa141542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:36.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 -- 172.21.15.38:0/4235786864 >> 172.21.15.38:0/4235786864 conn(0x7ffa140ce990 msgr2=0x7ffa1406c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:36.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 -- 172.21.15.38:0/4235786864 shutdown_connections 2024-08-15T17:58:36.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:36.906+0000 7ffa1ad95700 1 -- 172.21.15.38:0/4235786864 wait complete. 2024-08-15T17:58:36.919 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-08-15T17:58:37.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.026+0000 7fa0aa294700 1 Processor -- start 2024-08-15T17:58:37.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.026+0000 7fa0aa294700 1 -- start start 2024-08-15T17:58:37.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa0a4067030 0x7fa0a4067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:37.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa0a40cfb00 0x7fa0a40cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:37.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a40cf670 con 0x7fa0a40cb5c0 2024-08-15T17:58:37.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a40cea60 con 0x7fa0a40cfb00 2024-08-15T17:58:37.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a40ced40 con 0x7fa0a4067030 2024-08-15T17:58:37.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0a8a91700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa0a40cfb00 0x7fa0a40cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:37.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0a8a91700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa0a40cfb00 0x7fa0a40cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53486/0 (socket says 172.21.15.38:53486) 2024-08-15T17:58:37.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0a8a91700 1 -- 172.21.15.38:0/1290240075 learned_addr learned my addr 172.21.15.38:0/1290240075 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:37.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0a9292700 1 --2- 172.21.15.38:0/1290240075 >> v2:172.21.15.104:3300/0 conn(0x7fa0a4067030 0x7fa0a4067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:37.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0a8a91700 1 -- 172.21.15.38:0/1290240075 >> v2:172.21.15.104:3300/0 conn(0x7fa0a4067030 msgr2=0x7fa0a4067400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:37.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0a8a91700 1 --2- 172.21.15.38:0/1290240075 >> v2:172.21.15.104:3300/0 conn(0x7fa0a4067030 0x7fa0a4067400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:37.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0a8a91700 1 -- 172.21.15.38:0/1290240075 >> v1:172.21.15.104:6789/0 conn(0x7fa0a40cb5c0 legacy=0x7fa0a4158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:37.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0a8a91700 1 -- 172.21.15.38:0/1290240075 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a40cf020 con 0x7fa0a40cfb00 2024-08-15T17:58:37.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0a8a91700 1 --2- 172.21.15.38:0/1290240075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a40cfb00 0x7fa0a40cb080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40d1070 tx=0x7fa094009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89649f9bb4c50dec server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:37.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa09bfff700 1 -- 172.21.15.38:0/1290240075 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09400b860 con 0x7fa0a40cfb00 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa09bfff700 1 -- 172.21.15.38:0/1290240075 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09400b9c0 con 0x7fa0a40cfb00 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa09bfff700 1 -- 172.21.15.38:0/1290240075 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09400bd60 con 0x7fa0a40cfb00 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 -- 172.21.15.38:0/1290240075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a40cfb00 msgr2=0x7fa0a40cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 --2- 172.21.15.38:0/1290240075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a40cfb00 0x7fa0a40cb080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40d1070 tx=0x7fa094009250 comp rx=0 tx=0).stop 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 -- 172.21.15.38:0/1290240075 shutdown_connections 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 --2- 172.21.15.38:0/1290240075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a40cfb00 0x7fa0a40cb080 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 --2- 172.21.15.38:0/1290240075 >> v2:172.21.15.104:3300/0 conn(0x7fa0a4067030 0x7fa0a4067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 -- 172.21.15.38:0/1290240075 >> 172.21.15.38:0/1290240075 conn(0x7fa0a40d0b50 msgr2=0x7fa0a40ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 -- 172.21.15.38:0/1290240075 shutdown_connections 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 -- 172.21.15.38:0/1290240075 wait complete. 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 Processor -- start 2024-08-15T17:58:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 -- start start 2024-08-15T17:58:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a4067030 0x7fa0a4157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0a40cb5c0 0x7fa0a4154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.030+0000 7fa0aa294700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a40cfb00 0x7fa0a413daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0aa294700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a414d740 con 0x7fa0a4067030 2024-08-15T17:58:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0aa294700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4157bc0 con 0x7fa0a40cfb00 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0aa294700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4155020 con 0x7fa0a40cb5c0 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a9292700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a4067030 0x7fa0a4157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a9292700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a4067030 0x7fa0a4157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53502/0 (socket says 172.21.15.38:53502) 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a9292700 1 -- 172.21.15.38:0/2722279748 learned_addr learned my addr 172.21.15.38:0/2722279748 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a9a93700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a40cfb00 0x7fa0a413daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a8a91700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0a40cb5c0 0x7fa0a4154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a9292700 1 -- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0a40cb5c0 msgr2=0x7fa0a4154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a9292700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0a40cb5c0 0x7fa0a4154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a9292700 1 -- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a40cfb00 msgr2=0x7fa0a413daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a9292700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a40cfb00 0x7fa0a413daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a9292700 1 -- 172.21.15.38:0/2722279748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0940094f0 con 0x7fa0a4067030 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a9a93700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a40cfb00 0x7fa0a413daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:58:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a8a91700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0a40cb5c0 0x7fa0a4154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:58:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0a9292700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a4067030 0x7fa0a4157680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000d8c0 tx=0x7fa0a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0997fa700 1 -- 172.21.15.38:0/2722279748 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000de60 con 0x7fa0a4067030 2024-08-15T17:58:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0997fa700 1 -- 172.21.15.38:0/2722279748 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a000b7e0 con 0x7fa0a4067030 2024-08-15T17:58:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0997fa700 1 -- 172.21.15.38:0/2722279748 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000bba0 con 0x7fa0a4067030 2024-08-15T17:58:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0aa294700 1 -- 172.21.15.38:0/2722279748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a413dfe0 con 0x7fa0a4067030 2024-08-15T17:58:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0aa294700 1 -- 172.21.15.38:0/2722279748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a4157e10 con 0x7fa0a4067030 2024-08-15T17:58:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0aa294700 1 -- 172.21.15.38:0/2722279748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a40638f0 con 0x7fa0a4067030 2024-08-15T17:58:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.034+0000 7fa0997fa700 1 -- 172.21.15.38:0/2722279748 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa0a001e460 con 0x7fa0a4067030 2024-08-15T17:58:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.038+0000 7fa0997fa700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa088060860 0x7fa088062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.038+0000 7fa0997fa700 1 -- 172.21.15.38:0/2722279748 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa0a007d390 con 0x7fa0a4067030 2024-08-15T17:58:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.038+0000 7fa0a8a91700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa088060860 0x7fa088062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.038+0000 7fa0a8a91700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa088060860 0x7fa088062d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40d1070 tx=0x7fa094017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.042+0000 7fa0997fa700 1 -- 172.21.15.38:0/2722279748 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa0a004d7e0 con 0x7fa0a4067030 2024-08-15T17:58:37.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.246+0000 7fa0aa294700 1 -- 172.21.15.38:0/2722279748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa0a40b0aa0 con 0x7fa0a4067030 2024-08-15T17:58:37.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.882+0000 7fa0997fa700 1 -- 172.21.15.38:0/2722279748 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa0a0050e00 con 0x7fa0a4067030 2024-08-15T17:58:37.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 -- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa088060860 msgr2=0x7fa088062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:37.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa088060860 0x7fa088062d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40d1070 tx=0x7fa094017000 comp rx=0 tx=0).stop 2024-08-15T17:58:37.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 -- 172.21.15.38:0/2722279748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a4067030 msgr2=0x7fa0a4157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:37.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a4067030 0x7fa0a4157680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000d8c0 tx=0x7fa0a0008040 comp rx=0 tx=0).stop 2024-08-15T17:58:37.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 -- 172.21.15.38:0/2722279748 shutdown_connections 2024-08-15T17:58:37.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa088060860 0x7fa088062d10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:37.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a40cfb00 0x7fa0a413daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:37.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0a40cb5c0 0x7fa0a4154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:37.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 --2- 172.21.15.38:0/2722279748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a4067030 0x7fa0a4157680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:37.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 -- 172.21.15.38:0/2722279748 >> 172.21.15.38:0/2722279748 conn(0x7fa0a40d0b50 msgr2=0x7fa0a40b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:37.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 -- 172.21.15.38:0/2722279748 shutdown_connections 2024-08-15T17:58:37.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:37.886+0000 7fa0aa294700 1 -- 172.21.15.38:0/2722279748 wait complete. 2024-08-15T17:58:37.905 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-08-15T17:58:38.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.018+0000 7fce55087700 1 Processor -- start 2024-08-15T17:58:38.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.018+0000 7fce55087700 1 -- start start 2024-08-15T17:58:38.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce55087700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce5006e240 0x7fce5006e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:38.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce55087700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fce500aec90 0x7fce500aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:38.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce55087700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce500ab3f0 con 0x7fce50158a90 2024-08-15T17:58:38.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce55087700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce50159600 con 0x7fce500aec90 2024-08-15T17:58:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce55087700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce500ad4a0 con 0x7fce5006e240 2024-08-15T17:58:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce54886700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fce50158a90 0x7fce50158ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40050/0 (socket says 172.21.15.38:40050) 2024-08-15T17:58:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce54886700 1 -- 172.21.15.38:0/1467014139 learned_addr learned my addr 172.21.15.38:0/1467014139 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4f7fe700 1 --2- 172.21.15.38:0/1467014139 >> v2:172.21.15.104:3300/0 conn(0x7fce500aec90 0x7fce500aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4ffff700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce5006e240 0x7fce5006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51476143 0 0) 0x7fce500ab3f0 con 0x7fce50158a90 2024-08-15T17:58:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce3c003570 con 0x7fce50158a90 2024-08-15T17:58:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2818448889 0 0) 0x7fce3c003570 con 0x7fce50158a90 2024-08-15T17:58:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce5006e240 msgr2=0x7fce5006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce5006e240 0x7fce5006e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 >> v2:172.21.15.104:3300/0 conn(0x7fce500aec90 msgr2=0x7fce500aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 --2- 172.21.15.38:0/1467014139 >> v2:172.21.15.104:3300/0 conn(0x7fce500aec90 0x7fce500aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce500ab6d0 con 0x7fce50158a90 2024-08-15T17:58:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7fce480017f0 con 0x7fce50158a90 2024-08-15T17:58:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 >> v1:172.21.15.38:6789/0 conn(0x7fce50158a90 legacy=0x7fce50158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 0x7fce3c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce5006e240 0x7fce3c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce3c004d50 0x7fce3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce50159600 con 0x7fce3c003a60 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500ad4a0 con 0x7fce3c004d50 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c00ba60 con 0x7fce5006e240 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4f7fe700 1 --2- 172.21.15.38:0/1467014139 >> v2:172.21.15.104:3300/0 conn(0x7fce500aec90 0x7fce500aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4ffff700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 0x7fce3c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4f7fe700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce5006e240 0x7fce3c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce54886700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce3c004d50 0x7fce3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4ffff700 1 -- 172.21.15.38:0/1467014139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce5006e240 msgr2=0x7fce3c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4ffff700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce5006e240 0x7fce3c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4ffff700 1 -- 172.21.15.38:0/1467014139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce3c004d50 msgr2=0x7fce3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4ffff700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce3c004d50 0x7fce3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4ffff700 1 -- 172.21.15.38:0/1467014139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c003570 con 0x7fce3c003a60 2024-08-15T17:58:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4ffff700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 0x7fce3c003e70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce38009c00 tx=0x7fce380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2bcc068211a429 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:38.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3800e6b0 con 0x7fce3c003a60 2024-08-15T17:58:38.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3800e810 con 0x7fce3c003a60 2024-08-15T17:58:38.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce4effd700 1 -- 172.21.15.38:0/1467014139 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3800ebd0 con 0x7fce3c003a60 2024-08-15T17:58:38.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce55087700 1 -- 172.21.15.38:0/1467014139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 msgr2=0x7fce3c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:38.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce55087700 1 --2- 172.21.15.38:0/1467014139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 0x7fce3c003e70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce38009c00 tx=0x7fce380091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce54886700 1 -- 172.21.15.38:0/1467014139 reap_dead start 2024-08-15T17:58:38.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce55087700 1 -- 172.21.15.38:0/1467014139 shutdown_connections 2024-08-15T17:58:38.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.022+0000 7fce55087700 1 -- 172.21.15.38:0/1467014139 >> 172.21.15.38:0/1467014139 conn(0x7fce500678a0 msgr2=0x7fce50067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:38.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 -- 172.21.15.38:0/1467014139 shutdown_connections 2024-08-15T17:58:38.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 -- 172.21.15.38:0/1467014139 wait complete. 2024-08-15T17:58:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 Processor -- start 2024-08-15T17:58:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 -- start start 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 0x7fce5014b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce501524c0 0x7fce500b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce500b07b0 0x7fce5006a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b1860 con 0x7fce3c003a60 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b0c50 con 0x7fce501524c0 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b0f30 con 0x7fce500b07b0 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce4ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 0x7fce5014b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce4ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 0x7fce5014b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53534/0 (socket says 172.21.15.38:53534) 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce4ffff700 1 -- 172.21.15.38:0/3548831867 learned_addr learned my addr 172.21.15.38:0/3548831867 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce4f7fe700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce501524c0 0x7fce500b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce54886700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce500b07b0 0x7fce5006a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce4ffff700 1 -- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce500b07b0 msgr2=0x7fce5006a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce4ffff700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce500b07b0 0x7fce5006a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce4ffff700 1 -- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce501524c0 msgr2=0x7fce500b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce4ffff700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce501524c0 0x7fce500b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce4ffff700 1 -- 172.21.15.38:0/3548831867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce500b1210 con 0x7fce3c003a60 2024-08-15T17:58:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce54886700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce500b07b0 0x7fce5006a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce4ffff700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 0x7fce5014b960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fce3c004730 tx=0x7fce3801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce47fff700 1 -- 172.21.15.38:0/3548831867 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38022070 con 0x7fce3c003a60 2024-08-15T17:58:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce47fff700 1 -- 172.21.15.38:0/3548831867 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3800e980 con 0x7fce3c003a60 2024-08-15T17:58:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 -- 172.21.15.38:0/3548831867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce500b14f0 con 0x7fce3c003a60 2024-08-15T17:58:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce47fff700 1 -- 172.21.15.38:0/3548831867 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38027470 con 0x7fce3c003a60 2024-08-15T17:58:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.026+0000 7fce55087700 1 -- 172.21.15.38:0/3548831867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5006a900 con 0x7fce3c003a60 2024-08-15T17:58:38.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.030+0000 7fce55087700 1 -- 172.21.15.38:0/3548831867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce50064330 con 0x7fce3c003a60 2024-08-15T17:58:38.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.030+0000 7fce47fff700 1 -- 172.21.15.38:0/3548831867 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fce3801c450 con 0x7fce3c003a60 2024-08-15T17:58:38.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.030+0000 7fce47fff700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fce30060860 0x7fce30062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:38.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.030+0000 7fce47fff700 1 -- 172.21.15.38:0/3548831867 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce3807e750 con 0x7fce3c003a60 2024-08-15T17:58:38.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.030+0000 7fce4f7fe700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fce30060860 0x7fce30062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.034+0000 7fce4f7fe700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fce30060860 0x7fce30062d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce500ab240 tx=0x7fce400091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.038+0000 7fce47fff700 1 -- 172.21.15.38:0/3548831867 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fce3804eba0 con 0x7fce3c003a60 2024-08-15T17:58:38.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.238+0000 7fce55087700 1 -- 172.21.15.38:0/3548831867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fce5006ed70 con 0x7fce3c003a60 2024-08-15T17:58:38.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.894+0000 7fce47fff700 1 -- 172.21.15.38:0/3548831867 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fce380170c0 con 0x7fce3c003a60 2024-08-15T17:58:38.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 -- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fce30060860 msgr2=0x7fce30062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:38.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fce30060860 0x7fce30062d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce500ab240 tx=0x7fce400091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 -- 172.21.15.38:0/3548831867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 msgr2=0x7fce5014b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:38.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 0x7fce5014b960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fce3c004730 tx=0x7fce3801a000 comp rx=0 tx=0).stop 2024-08-15T17:58:38.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 -- 172.21.15.38:0/3548831867 shutdown_connections 2024-08-15T17:58:38.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fce30060860 0x7fce30062d10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce500b07b0 0x7fce5006a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce501524c0 0x7fce500b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 --2- 172.21.15.38:0/3548831867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce3c003a60 0x7fce5014b960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:38.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 -- 172.21.15.38:0/3548831867 >> 172.21.15.38:0/3548831867 conn(0x7fce500678a0 msgr2=0x7fce50063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:38.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 -- 172.21.15.38:0/3548831867 shutdown_connections 2024-08-15T17:58:38.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:38.898+0000 7fce55087700 1 -- 172.21.15.38:0/3548831867 wait complete. 2024-08-15T17:58:38.920 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:58:39.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f9112b700 1 Processor -- start 2024-08-15T17:58:39.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f9112b700 1 -- start start 2024-08-15T17:58:39.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f9112b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5f8c156d40 0x7f5f8c157110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f9112b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c15ba70 con 0x7f5f8c15b650 2024-08-15T17:58:39.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f9112b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c0addb0 con 0x7f5f8c06d2f0 2024-08-15T17:58:39.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f9112b700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8c0ae930 con 0x7f5f8c156d40 2024-08-15T17:58:39.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f8ad9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5f8c156d40 0x7f5f8c157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f8b59e700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5f8c15b650 0x7f5f8c0cbbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40066/0 (socket says 172.21.15.38:40066) 2024-08-15T17:58:39.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f8b59e700 1 -- 172.21.15.38:0/815903048 learned_addr learned my addr 172.21.15.38:0/815903048 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:39.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f8ad9d700 1 -- 172.21.15.38:0/815903048 >> v1:172.21.15.104:6789/0 conn(0x7f5f8c06d2f0 legacy=0x7f5f8c15af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f8ad9d700 1 -- 172.21.15.38:0/815903048 >> v1:172.21.15.38:6789/0 conn(0x7f5f8c15b650 legacy=0x7f5f8c0cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f89d9b700 1 -- 172.21.15.38:0/815903048 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290151130 0 0) 0x7f5f84001490 con 0x7f5f8c15b650 2024-08-15T17:58:39.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.026+0000 7f5f8ad9d700 1 -- 172.21.15.38:0/815903048 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c0ae000 con 0x7f5f8c156d40 2024-08-15T17:58:39.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.030+0000 7f5f8ad9d700 1 --2- 172.21.15.38:0/815903048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f8c156d40 0x7f5f8c157110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0098e0 tx=0x7f5f7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc4eb0bb533850b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:39.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.030+0000 7f5f89d9b700 1 -- 172.21.15.38:0/815903048 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c009e80 con 0x7f5f8c156d40 2024-08-15T17:58:39.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.030+0000 7f5f89d9b700 1 -- 172.21.15.38:0/815903048 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c00b860 con 0x7f5f8c156d40 2024-08-15T17:58:39.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.030+0000 7f5f89d9b700 1 -- 172.21.15.38:0/815903048 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c00bbe0 con 0x7f5f8c156d40 2024-08-15T17:58:39.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.030+0000 7f5f9112b700 1 -- 172.21.15.38:0/815903048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f8c156d40 msgr2=0x7f5f8c157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.030+0000 7f5f9112b700 1 --2- 172.21.15.38:0/815903048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f8c156d40 0x7f5f8c157110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0098e0 tx=0x7f5f7c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.030+0000 7f5f9112b700 1 -- 172.21.15.38:0/815903048 shutdown_connections 2024-08-15T17:58:39.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.030+0000 7f5f9112b700 1 --2- 172.21.15.38:0/815903048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f8c156d40 0x7f5f8c157110 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.030+0000 7f5f9112b700 1 -- 172.21.15.38:0/815903048 >> 172.21.15.38:0/815903048 conn(0x7f5f8c0ceb30 msgr2=0x7f5f8c06cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:39.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.030+0000 7f5f9112b700 1 -- 172.21.15.38:0/815903048 shutdown_connections 2024-08-15T17:58:39.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.030+0000 7f5f9112b700 1 -- 172.21.15.38:0/815903048 wait complete. 2024-08-15T17:58:39.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f9112b700 1 Processor -- start 2024-08-15T17:58:39.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f9112b700 1 -- start start 2024-08-15T17:58:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f9112b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f8c06d2f0 0x7f5f8c155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f9112b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8c156d40 0x7f5f8c1672d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f9112b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f8c15b650 0x7f5f8c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f9112b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c167810 con 0x7f5f8c15b650 2024-08-15T17:58:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f9112b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c1682b0 con 0x7f5f8c156d40 2024-08-15T17:58:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f9112b700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c152960 con 0x7f5f8c06d2f0 2024-08-15T17:58:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f8b59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f8c15b650 0x7f5f8c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f8b59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f8c15b650 0x7f5f8c154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53558/0 (socket says 172.21.15.38:53558) 2024-08-15T17:58:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f8b59e700 1 -- 172.21.15.38:0/2046883526 learned_addr learned my addr 172.21.15.38:0/2046883526 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f8ad9d700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f8c06d2f0 0x7f5f8c155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f8a59c700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8c156d40 0x7f5f8c1672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f8b59e700 1 -- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f8c06d2f0 msgr2=0x7f5f8c155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f8b59e700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f8c06d2f0 0x7f5f8c155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f8b59e700 1 -- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8c156d40 msgr2=0x7f5f8c1672d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f8b59e700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8c156d40 0x7f5f8c1672d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f8b59e700 1 -- 172.21.15.38:0/2046883526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c009420 con 0x7f5f8c15b650 2024-08-15T17:58:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f8b59e700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f8c15b650 0x7f5f8c154330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400ccc0 tx=0x7f5f84001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f82ffd700 1 -- 172.21.15.38:0/2046883526 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84004bb0 con 0x7f5f8c15b650 2024-08-15T17:58:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f82ffd700 1 -- 172.21.15.38:0/2046883526 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f84004d10 con 0x7f5f8c15b650 2024-08-15T17:58:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f9112b700 1 -- 172.21.15.38:0/2046883526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c0c3aa0 con 0x7f5f8c15b650 2024-08-15T17:58:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f82ffd700 1 -- 172.21.15.38:0/2046883526 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400aaf0 con 0x7f5f8c15b650 2024-08-15T17:58:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.034+0000 7f5f9112b700 1 -- 172.21.15.38:0/2046883526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c167cf0 con 0x7f5f8c15b650 2024-08-15T17:58:39.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.038+0000 7f5f9112b700 1 -- 172.21.15.38:0/2046883526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c0645c0 con 0x7f5f8c15b650 2024-08-15T17:58:39.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.038+0000 7f5f82ffd700 1 -- 172.21.15.38:0/2046883526 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5f840070f0 con 0x7f5f8c15b650 2024-08-15T17:58:39.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.038+0000 7f5f82ffd700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f6c060860 0x7f5f6c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.038+0000 7f5f82ffd700 1 -- 172.21.15.38:0/2046883526 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f8407c860 con 0x7f5f8c15b650 2024-08-15T17:58:39.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.038+0000 7f5f8ad9d700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f6c060860 0x7f5f6c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.042+0000 7f5f8ad9d700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f6c060860 0x7f5f6c062d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0093f0 tx=0x7f5f7c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:39.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.046+0000 7f5f82ffd700 1 -- 172.21.15.38:0/2046883526 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5f8404cc90 con 0x7f5f8c15b650 2024-08-15T17:58:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.246+0000 7f5f9112b700 1 -- 172.21.15.38:0/2046883526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f8c15ba70 con 0x7f5f8c15b650 2024-08-15T17:58:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.246+0000 7f5f82ffd700 1 -- 172.21.15.38:0/2046883526 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f5f8c15ba70 con 0x7f5f8c15b650 2024-08-15T17:58:39.250 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:39.250 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2444719791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2444719791},{"type":"v1","addr":"172.21.15.38:6833","nonce":2444719791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2667173752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2667173752},{"type":"v1","addr":"172.21.15.38:6835","nonce":2667173752}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1185013072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1185013072},{"type":"v1","addr":"172.21.15.104:6835","nonce":1185013072}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:58:35.622492+0000","modified":"2024-08-15T17:58:38.885045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/3628618356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3628618356},{"type":"v1","addr":"172.21.15.104:6837","nonce":3628618356}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:58:39.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 -- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f6c060860 msgr2=0x7f5f6c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f6c060860 0x7f5f6c062d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0093f0 tx=0x7f5f7c017000 comp rx=0 tx=0).stop 2024-08-15T17:58:39.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 -- 172.21.15.38:0/2046883526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f8c15b650 msgr2=0x7f5f8c154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f8c15b650 0x7f5f8c154330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400ccc0 tx=0x7f5f84001a50 comp rx=0 tx=0).stop 2024-08-15T17:58:39.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 -- 172.21.15.38:0/2046883526 shutdown_connections 2024-08-15T17:58:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f6c060860 0x7f5f6c062d10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f8c15b650 0x7f5f8c154330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8c156d40 0x7f5f8c1672d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 --2- 172.21.15.38:0/2046883526 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f8c06d2f0 0x7f5f8c155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 -- 172.21.15.38:0/2046883526 >> 172.21.15.38:0/2046883526 conn(0x7f5f8c0ceb30 msgr2=0x7f5f8c150870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 -- 172.21.15.38:0/2046883526 shutdown_connections 2024-08-15T17:58:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.250+0000 7f5f9112b700 1 -- 172.21.15.38:0/2046883526 wait complete. 2024-08-15T17:58:39.254 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-08-15T17:58:39.264 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbabfff700 1 Processor -- start 2024-08-15T17:58:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbabfff700 1 -- start start 2024-08-15T17:58:39.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbabfff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdbac153320 0x7fdbac1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbabfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbac15a190 0x7fdbac15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbabfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdbac0ccaa0 0x7fdbac0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbabfff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbac0ce430 con 0x7fdbac0ccaa0 2024-08-15T17:58:39.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbabfff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbac0cd440 con 0x7fdbac153320 2024-08-15T17:58:39.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbabfff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac15dbd0 con 0x7fdbac15a190 2024-08-15T17:58:39.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbab7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdbac0ccaa0 0x7fdbac0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbab7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdbac0ccaa0 0x7fdbac0ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53570/0 (socket says 172.21.15.38:53570) 2024-08-15T17:58:39.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbab7fe700 1 -- 172.21.15.38:0/1097453899 learned_addr learned my addr 172.21.15.38:0/1097453899 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:39.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbaaffd700 1 --2- 172.21.15.38:0/1097453899 >> v2:172.21.15.104:3300/0 conn(0x7fdbac153320 0x7fdbac1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbaa7fc700 1 --2- 172.21.15.38:0/1097453899 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbac15a190 0x7fdbac15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbab7fe700 1 -- 172.21.15.38:0/1097453899 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbac15a190 msgr2=0x7fdbac15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbab7fe700 1 --2- 172.21.15.38:0/1097453899 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbac15a190 0x7fdbac15cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbab7fe700 1 -- 172.21.15.38:0/1097453899 >> v2:172.21.15.104:3300/0 conn(0x7fdbac153320 msgr2=0x7fdbac1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbab7fe700 1 --2- 172.21.15.38:0/1097453899 >> v2:172.21.15.104:3300/0 conn(0x7fdbac153320 0x7fdbac1536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbab7fe700 1 -- 172.21.15.38:0/1097453899 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac15d370 con 0x7fdbac0ccaa0 2024-08-15T17:58:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbaa7fc700 1 --2- 172.21.15.38:0/1097453899 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbac15a190 0x7fdbac15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdbab7fe700 1 --2- 172.21.15.38:0/1097453899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbac0ccaa0 0x7fdbac0ccf00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0d0cd0 tx=0x7fdba4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365603ef38e16cc4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdba9ffb700 1 -- 172.21.15.38:0/1097453899 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4009e70 con 0x7fdbac0ccaa0 2024-08-15T17:58:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdba9ffb700 1 -- 172.21.15.38:0/1097453899 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba400b860 con 0x7fdbac0ccaa0 2024-08-15T17:58:39.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.366+0000 7fdba9ffb700 1 -- 172.21.15.38:0/1097453899 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400bc00 con 0x7fdbac0ccaa0 2024-08-15T17:58:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.370+0000 7fdbabfff700 1 -- 172.21.15.38:0/1097453899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbac0ccaa0 msgr2=0x7fdbac0ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.370+0000 7fdbabfff700 1 --2- 172.21.15.38:0/1097453899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbac0ccaa0 0x7fdbac0ccf00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0d0cd0 tx=0x7fdba4009250 comp rx=0 tx=0).stop 2024-08-15T17:58:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.370+0000 7fdbabfff700 1 -- 172.21.15.38:0/1097453899 shutdown_connections 2024-08-15T17:58:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.370+0000 7fdbabfff700 1 --2- 172.21.15.38:0/1097453899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbac0ccaa0 0x7fdbac0ccf00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.370+0000 7fdbabfff700 1 --2- 172.21.15.38:0/1097453899 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbac15a190 0x7fdbac15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.370+0000 7fdbabfff700 1 --2- 172.21.15.38:0/1097453899 >> v2:172.21.15.104:3300/0 conn(0x7fdbac153320 0x7fdbac1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.370+0000 7fdbabfff700 1 -- 172.21.15.38:0/1097453899 >> 172.21.15.38:0/1097453899 conn(0x7fdbac0d00a0 msgr2=0x7fdbac0d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:39.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.370+0000 7fdbabfff700 1 -- 172.21.15.38:0/1097453899 shutdown_connections 2024-08-15T17:58:39.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.370+0000 7fdbabfff700 1 -- 172.21.15.38:0/1097453899 wait complete. 2024-08-15T17:58:39.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.370+0000 7fdbabfff700 1 Processor -- start 2024-08-15T17:58:39.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.370+0000 7fdbabfff700 1 -- start start 2024-08-15T17:58:39.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbabfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ccaa0 0x7fdbac152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbabfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbac153320 0x7fdbac0c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbabfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbac15a190 0x7fdbac158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbabfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac159460 con 0x7fdbac15a190 2024-08-15T17:58:39.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbabfff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0c7ac0 con 0x7fdbac0ccaa0 2024-08-15T17:58:39.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbabfff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0c8670 con 0x7fdbac153320 2024-08-15T17:58:39.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbab7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbac15a190 0x7fdbac158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbab7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbac15a190 0x7fdbac158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53572/0 (socket says 172.21.15.38:53572) 2024-08-15T17:58:39.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbab7fe700 1 -- 172.21.15.38:0/3037526553 learned_addr learned my addr 172.21.15.38:0/3037526553 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:39.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbaa7fc700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbac153320 0x7fdbac0c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbaaffd700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ccaa0 0x7fdbac152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbab7fe700 1 -- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbac153320 msgr2=0x7fdbac0c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbab7fe700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbac153320 0x7fdbac0c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbab7fe700 1 -- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ccaa0 msgr2=0x7fdbac152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbab7fe700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ccaa0 0x7fdbac152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbab7fe700 1 -- 172.21.15.38:0/3037526553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba40094f0 con 0x7fdbac15a190 2024-08-15T17:58:39.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbab7fe700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbac15a190 0x7fdbac158f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdba40090b0 tx=0x7fdba4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:39.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdba2ffd700 1 -- 172.21.15.38:0/3037526553 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400bbe0 con 0x7fdbac15a190 2024-08-15T17:58:39.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdba2ffd700 1 -- 172.21.15.38:0/3037526553 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba401a450 con 0x7fdbac15a190 2024-08-15T17:58:39.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbabfff700 1 -- 172.21.15.38:0/3037526553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac0c77e0 con 0x7fdbac15a190 2024-08-15T17:58:39.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdba2ffd700 1 -- 172.21.15.38:0/3037526553 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba401a7f0 con 0x7fdbac15a190 2024-08-15T17:58:39.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbabfff700 1 -- 172.21.15.38:0/3037526553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac0c8300 con 0x7fdbac15a190 2024-08-15T17:58:39.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.374+0000 7fdbabfff700 1 -- 172.21.15.38:0/3037526553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbac0c8800 con 0x7fdbac15a190 2024-08-15T17:58:39.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.378+0000 7fdba2ffd700 1 -- 172.21.15.38:0/3037526553 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdba401ab60 con 0x7fdbac15a190 2024-08-15T17:58:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.378+0000 7fdba2ffd700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb90060860 0x7fdb90062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.378+0000 7fdba2ffd700 1 -- 172.21.15.38:0/3037526553 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdba407e1c0 con 0x7fdbac15a190 2024-08-15T17:58:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.378+0000 7fdbaaffd700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb90060860 0x7fdb90062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.378+0000 7fdbaaffd700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb90060860 0x7fdb90062d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00f570 tx=0x7fdb9c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.386+0000 7fdba2ffd700 1 -- 172.21.15.38:0/3037526553 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdba404e570 con 0x7fdbac15a190 2024-08-15T17:58:39.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.554+0000 7fdbabfff700 1 -- 172.21.15.38:0/3037526553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbac0638f0 con 0x7fdbac15a190 2024-08-15T17:58:39.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.554+0000 7fdba2ffd700 1 -- 172.21.15.38:0/3037526553 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fdba4051b90 con 0x7fdbac15a190 2024-08-15T17:58:39.558 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:39.558 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:39.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 -- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb90060860 msgr2=0x7fdb90062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb90060860 0x7fdb90062d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00f570 tx=0x7fdb9c00b040 comp rx=0 tx=0).stop 2024-08-15T17:58:39.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 -- 172.21.15.38:0/3037526553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbac15a190 msgr2=0x7fdbac158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbac15a190 0x7fdbac158f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdba40090b0 tx=0x7fdba4017000 comp rx=0 tx=0).stop 2024-08-15T17:58:39.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 -- 172.21.15.38:0/3037526553 shutdown_connections 2024-08-15T17:58:39.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdb90060860 0x7fdb90062d10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbac15a190 0x7fdbac158f20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbac153320 0x7fdbac0c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 --2- 172.21.15.38:0/3037526553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ccaa0 0x7fdbac152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 -- 172.21.15.38:0/3037526553 >> 172.21.15.38:0/3037526553 conn(0x7fdbac0d00a0 msgr2=0x7fdbac0cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:39.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 -- 172.21.15.38:0/3037526553 shutdown_connections 2024-08-15T17:58:39.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.558+0000 7fdbabfff700 1 -- 172.21.15.38:0/3037526553 wait complete. 2024-08-15T17:58:39.573 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-08-15T17:58:39.573 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:58:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f895915f700 1 Processor -- start 2024-08-15T17:58:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f895915f700 1 -- start start 2024-08-15T17:58:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f895915f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f895413fa40 0x7f89540ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f895915f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f89540ae340 0x7f895406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f895915f700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89540ae8c0 con 0x7f895413f670 2024-08-15T17:58:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f895915f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89540b1820 con 0x7f89540ae340 2024-08-15T17:58:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f895915f700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89540b23c0 con 0x7f895413fa40 2024-08-15T17:58:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f8953fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f895413f670 0x7f89540b1cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40072/0 (socket says 172.21.15.38:40072) 2024-08-15T17:58:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f8953fff700 1 -- 172.21.15.38:0/2195357296 learned_addr learned my addr 172.21.15.38:0/2195357296 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f89537fe700 1 --2- 172.21.15.38:0/2195357296 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f895413fa40 0x7f89540ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f895895e700 1 --2- 172.21.15.38:0/2195357296 >> v2:172.21.15.104:3300/0 conn(0x7f89540ae340 0x7f895406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f8952ffd700 1 -- 172.21.15.38:0/2195357296 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822491995 0 0) 0x7f89540ae8c0 con 0x7f895413f670 2024-08-15T17:58:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f8952ffd700 1 -- 172.21.15.38:0/2195357296 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8934003570 con 0x7f895413f670 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f89537fe700 1 -- 172.21.15.38:0/2195357296 >> v2:172.21.15.104:3300/0 conn(0x7f89540ae340 msgr2=0x7f895406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f89537fe700 1 --2- 172.21.15.38:0/2195357296 >> v2:172.21.15.104:3300/0 conn(0x7f89540ae340 0x7f895406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f89537fe700 1 -- 172.21.15.38:0/2195357296 >> v1:172.21.15.38:6789/0 conn(0x7f895413f670 legacy=0x7f89540b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f8952ffd700 1 -- 172.21.15.38:0/2195357296 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1296636240 0 0) 0x7f8934003570 con 0x7f895413f670 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f89537fe700 1 -- 172.21.15.38:0/2195357296 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89540ad710 con 0x7f895413fa40 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f89537fe700 1 --2- 172.21.15.38:0/2195357296 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f895413fa40 0x7f89540ade00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f894c009e60 tx=0x7f894c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28dc631516691048 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f8952ffd700 1 -- 172.21.15.38:0/2195357296 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c001470 con 0x7f895413fa40 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f8952ffd700 1 -- 172.21.15.38:0/2195357296 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c00b7a0 con 0x7f895413fa40 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.678+0000 7f8952ffd700 1 -- 172.21.15.38:0/2195357296 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c00bb20 con 0x7f895413fa40 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 -- 172.21.15.38:0/2195357296 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f895413fa40 msgr2=0x7f89540ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 --2- 172.21.15.38:0/2195357296 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f895413fa40 0x7f89540ade00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f894c009e60 tx=0x7f894c009780 comp rx=0 tx=0).stop 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 -- 172.21.15.38:0/2195357296 shutdown_connections 2024-08-15T17:58:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 --2- 172.21.15.38:0/2195357296 >> v2:172.21.15.104:3300/0 conn(0x7f89540ae340 0x7f895406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 --2- 172.21.15.38:0/2195357296 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f895413fa40 0x7f89540ade00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 -- 172.21.15.38:0/2195357296 >> 172.21.15.38:0/2195357296 conn(0x7f89540ca2b0 msgr2=0x7f895406e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 -- 172.21.15.38:0/2195357296 shutdown_connections 2024-08-15T17:58:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 -- 172.21.15.38:0/2195357296 wait complete. 2024-08-15T17:58:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 Processor -- start 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 -- start start 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89540ae340 0x7f89540ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f895413f670 0x7f89541582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895413fa40 0x7f895406b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954158040 con 0x7f895413fa40 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89540ac560 con 0x7f89540ae340 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8954156760 con 0x7f895413f670 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895895e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895413fa40 0x7f895406b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f89537fe700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f895413f670 0x7f89541582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895895e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895413fa40 0x7f895406b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53576/0 (socket says 172.21.15.38:53576) 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895895e700 1 -- 172.21.15.38:0/3148396335 learned_addr learned my addr 172.21.15.38:0/3148396335 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f8953fff700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89540ae340 0x7f89540ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895895e700 1 -- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f895413f670 msgr2=0x7f89541582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895895e700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f895413f670 0x7f89541582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895895e700 1 -- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89540ae340 msgr2=0x7f89540ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895895e700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89540ae340 0x7f89540ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895895e700 1 -- 172.21.15.38:0/3148396335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c009a20 con 0x7f895413fa40 2024-08-15T17:58:39.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f8953fff700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89540ae340 0x7f89540ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:39.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895895e700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895413fa40 0x7f895406b290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f894400e8d0 tx=0x7f89440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f894bfff700 1 -- 172.21.15.38:0/3148396335 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944013680 con 0x7f895413fa40 2024-08-15T17:58:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f894bfff700 1 -- 172.21.15.38:0/3148396335 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89440137e0 con 0x7f895413fa40 2024-08-15T17:58:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 -- 172.21.15.38:0/3148396335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8954156a40 con 0x7f895413fa40 2024-08-15T17:58:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f894bfff700 1 -- 172.21.15.38:0/3148396335 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944013ba0 con 0x7f895413fa40 2024-08-15T17:58:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.682+0000 7f895915f700 1 -- 172.21.15.38:0/3148396335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8954157000 con 0x7f895413fa40 2024-08-15T17:58:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.686+0000 7f895915f700 1 -- 172.21.15.38:0/3148396335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89540643a0 con 0x7f895413fa40 2024-08-15T17:58:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.686+0000 7f894bfff700 1 -- 172.21.15.38:0/3148396335 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f89440123d0 con 0x7f895413fa40 2024-08-15T17:58:39.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.686+0000 7f894bfff700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8938060890 0x7f8938062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:39.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.686+0000 7f894bfff700 1 -- 172.21.15.38:0/3148396335 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f894407ccc0 con 0x7f895413fa40 2024-08-15T17:58:39.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.686+0000 7f8953fff700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8938060890 0x7f8938062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:39.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.686+0000 7f8953fff700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8938060890 0x7f8938062d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f893c0011e0 tx=0x7f893c001940 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:39.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.694+0000 7f894bfff700 1 -- 172.21.15.38:0/3148396335 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f894404d110 con 0x7f895413fa40 2024-08-15T17:58:39.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.854+0000 7f895915f700 1 -- 172.21.15.38:0/3148396335 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89540ad090 con 0x7f8938060890 2024-08-15T17:58:39.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.874+0000 7f894bfff700 1 -- 172.21.15.38:0/3148396335 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259018 (secure 0 0 0) 0x7f89540ad090 con 0x7f8938060890 2024-08-15T17:58:39.877 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.874+0000 7f895915f700 1 -- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8938060890 msgr2=0x7f8938062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.874+0000 7f895915f700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8938060890 0x7f8938062d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f893c0011e0 tx=0x7f893c001940 comp rx=0 tx=0).stop 2024-08-15T17:58:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.878+0000 7f895915f700 1 -- 172.21.15.38:0/3148396335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895413fa40 msgr2=0x7f895406b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.878+0000 7f895915f700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895413fa40 0x7f895406b290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f894400e8d0 tx=0x7f89440091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.878+0000 7f895915f700 1 -- 172.21.15.38:0/3148396335 shutdown_connections 2024-08-15T17:58:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.878+0000 7f895915f700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8938060890 0x7f8938062d40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.878+0000 7f895915f700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895413fa40 0x7f895406b290 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.878+0000 7f895915f700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f895413f670 0x7f89541582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.878+0000 7f895915f700 1 --2- 172.21.15.38:0/3148396335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89540ae340 0x7f89540ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.878+0000 7f895915f700 1 -- 172.21.15.38:0/3148396335 >> 172.21.15.38:0/3148396335 conn(0x7f89540ca2b0 msgr2=0x7f895413e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.878+0000 7f895915f700 1 -- 172.21.15.38:0/3148396335 shutdown_connections 2024-08-15T17:58:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:39.878+0000 7f895915f700 1 -- 172.21.15.38:0/3148396335 wait complete. 2024-08-15T17:58:39.881 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-08-15T17:58:39.892 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-08-15T17:58:38.168268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":186,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":124,"acting":124,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2390404,"kb_used_data":2588,"kb_used_omap":13,"kb_used_meta":2387058,"kb_avail":752584316,"statfs":{"total":773094113280,"available":770646339584,"internally_reserved":0,"allocated":2650112,"data_stored":1644623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2444348096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001414"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286540+0000","last_change":"2024-08-15T17:58:36.286540+0000","last_active":"2024-08-15T17:58:36.286540+0000","last_peered":"2024-08-15T17:58:36.286540+0000","last_clean":"2024-08-15T17:58:36.286540+0000","last_became_active":"2024-08-15T17:58:36.285657+0000","last_became_peered":"2024-08-15T17:58:36.285657+0000","last_unstale":"2024-08-15T17:58:36.286540+0000","last_undegraded":"2024-08-15T17:58:36.286540+0000","last_fullsized":"2024-08-15T17:58:36.286540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:50:39.313237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.279817+0000","last_change":"2024-08-15T17:58:36.279817+0000","last_active":"2024-08-15T17:58:36.279817+0000","last_peered":"2024-08-15T17:58:36.279817+0000","last_clean":"2024-08-15T17:58:36.279817+0000","last_became_active":"2024-08-15T17:58:36.279494+0000","last_became_peered":"2024-08-15T17:58:36.279494+0000","last_unstale":"2024-08-15T17:58:36.279817+0000","last_undegraded":"2024-08-15T17:58:36.279817+0000","last_fullsized":"2024-08-15T17:58:36.279817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:48:29.832597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285873+0000","last_change":"2024-08-15T17:58:36.285873+0000","last_active":"2024-08-15T17:58:36.285873+0000","last_peered":"2024-08-15T17:58:36.285873+0000","last_clean":"2024-08-15T17:58:36.285873+0000","last_became_active":"2024-08-15T17:58:36.285630+0000","last_became_peered":"2024-08-15T17:58:36.285630+0000","last_unstale":"2024-08-15T17:58:36.285873+0000","last_undegraded":"2024-08-15T17:58:36.285873+0000","last_fullsized":"2024-08-15T17:58:36.285873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:30:51.740686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286578+0000","last_change":"2024-08-15T17:58:36.286578+0000","last_active":"2024-08-15T17:58:36.286578+0000","last_peered":"2024-08-15T17:58:36.286578+0000","last_clean":"2024-08-15T17:58:36.286578+0000","last_became_active":"2024-08-15T17:58:36.286298+0000","last_became_peered":"2024-08-15T17:58:36.286298+0000","last_unstale":"2024-08-15T17:58:36.286578+0000","last_undegraded":"2024-08-15T17:58:36.286578+0000","last_fullsized":"2024-08-15T17:58:36.286578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:47:17.486762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285830+0000","last_change":"2024-08-15T17:58:36.285830+0000","last_active":"2024-08-15T17:58:36.285830+0000","last_peered":"2024-08-15T17:58:36.285830+0000","last_clean":"2024-08-15T17:58:36.285830+0000","last_became_active":"2024-08-15T17:58:36.285422+0000","last_became_peered":"2024-08-15T17:58:36.285422+0000","last_unstale":"2024-08-15T17:58:36.285830+0000","last_undegraded":"2024-08-15T17:58:36.285830+0000","last_fullsized":"2024-08-15T17:58:36.285830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:15:59.759500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286478+0000","last_change":"2024-08-15T17:58:36.286478+0000","last_active":"2024-08-15T17:58:36.286478+0000","last_peered":"2024-08-15T17:58:36.286478+0000","last_clean":"2024-08-15T17:58:36.286478+0000","last_became_active":"2024-08-15T17:58:36.285967+0000","last_became_peered":"2024-08-15T17:58:36.285967+0000","last_unstale":"2024-08-15T17:58:36.286478+0000","last_undegraded":"2024-08-15T17:58:36.286478+0000","last_fullsized":"2024-08-15T17:58:36.286478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.279946+0000","last_change":"2024-08-15T17:58:36.279946+0000","last_active":"2024-08-15T17:58:36.279946+0000","last_peered":"2024-08-15T17:58:36.279946+0000","last_clean":"2024-08-15T17:58:36.279946+0000","last_became_active":"2024-08-15T17:58:36.279648+0000","last_became_peered":"2024-08-15T17:58:36.279648+0000","last_unstale":"2024-08-15T17:58:36.279946+0000","last_undegraded":"2024-08-15T17:58:36.279946+0000","last_fullsized":"2024-08-15T17:58:36.279946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:34:14.015831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286890+0000","last_change":"2024-08-15T17:58:36.286890+0000","last_active":"2024-08-15T17:58:36.286890+0000","last_peered":"2024-08-15T17:58:36.286890+0000","last_clean":"2024-08-15T17:58:36.286890+0000","last_became_active":"2024-08-15T17:58:36.286332+0000","last_became_peered":"2024-08-15T17:58:36.286332+0000","last_unstale":"2024-08-15T17:58:36.286890+0000","last_undegraded":"2024-08-15T17:58:36.286890+0000","last_fullsized":"2024-08-15T17:58:36.286890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:07:28.572500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.287994+0000","last_change":"2024-08-15T17:58:36.287994+0000","last_active":"2024-08-15T17:58:36.287994+0000","last_peered":"2024-08-15T17:58:36.287994+0000","last_clean":"2024-08-15T17:58:36.287994+0000","last_became_active":"2024-08-15T17:58:36.286645+0000","last_became_peered":"2024-08-15T17:58:36.286645+0000","last_unstale":"2024-08-15T17:58:36.287994+0000","last_undegraded":"2024-08-15T17:58:36.287994+0000","last_fullsized":"2024-08-15T17:58:36.287994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280114+0000","last_change":"2024-08-15T17:58:36.280114+0000","last_active":"2024-08-15T17:58:36.280114+0000","last_peered":"2024-08-15T17:58:36.280114+0000","last_clean":"2024-08-15T17:58:36.280114+0000","last_became_active":"2024-08-15T17:58:36.279785+0000","last_became_peered":"2024-08-15T17:58:36.279785+0000","last_unstale":"2024-08-15T17:58:36.280114+0000","last_undegraded":"2024-08-15T17:58:36.280114+0000","last_fullsized":"2024-08-15T17:58:36.280114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:56:03.150852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290521+0000","last_change":"2024-08-15T17:58:36.290521+0000","last_active":"2024-08-15T17:58:36.290521+0000","last_peered":"2024-08-15T17:58:36.290521+0000","last_clean":"2024-08-15T17:58:36.290521+0000","last_became_active":"2024-08-15T17:58:36.290154+0000","last_became_peered":"2024-08-15T17:58:36.290154+0000","last_unstale":"2024-08-15T17:58:36.290521+0000","last_undegraded":"2024-08-15T17:58:36.290521+0000","last_fullsized":"2024-08-15T17:58:36.290521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:45.183805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280839+0000","last_change":"2024-08-15T17:58:36.280839+0000","last_active":"2024-08-15T17:58:36.280839+0000","last_peered":"2024-08-15T17:58:36.280839+0000","last_clean":"2024-08-15T17:58:36.280839+0000","last_became_active":"2024-08-15T17:58:36.280519+0000","last_became_peered":"2024-08-15T17:58:36.280519+0000","last_unstale":"2024-08-15T17:58:36.280839+0000","last_undegraded":"2024-08-15T17:58:36.280839+0000","last_fullsized":"2024-08-15T17:58:36.280839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.278583+0000","last_change":"2024-08-15T17:58:36.278583+0000","last_active":"2024-08-15T17:58:36.278583+0000","last_peered":"2024-08-15T17:58:36.278583+0000","last_clean":"2024-08-15T17:58:36.278583+0000","last_became_active":"2024-08-15T17:58:36.278308+0000","last_became_peered":"2024-08-15T17:58:36.278308+0000","last_unstale":"2024-08-15T17:58:36.278583+0000","last_undegraded":"2024-08-15T17:58:36.278583+0000","last_fullsized":"2024-08-15T17:58:36.278583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:56:49.069407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283003+0000","last_change":"2024-08-15T17:58:36.283003+0000","last_active":"2024-08-15T17:58:36.283003+0000","last_peered":"2024-08-15T17:58:36.283003+0000","last_clean":"2024-08-15T17:58:36.283003+0000","last_became_active":"2024-08-15T17:58:36.282691+0000","last_became_peered":"2024-08-15T17:58:36.282691+0000","last_unstale":"2024-08-15T17:58:36.283003+0000","last_undegraded":"2024-08-15T17:58:36.283003+0000","last_fullsized":"2024-08-15T17:58:36.283003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:48:28.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285479+0000","last_change":"2024-08-15T17:58:36.285479+0000","last_active":"2024-08-15T17:58:36.285479+0000","last_peered":"2024-08-15T17:58:36.285479+0000","last_clean":"2024-08-15T17:58:36.285479+0000","last_became_active":"2024-08-15T17:58:36.285044+0000","last_became_peered":"2024-08-15T17:58:36.285044+0000","last_unstale":"2024-08-15T17:58:36.285479+0000","last_undegraded":"2024-08-15T17:58:36.285479+0000","last_fullsized":"2024-08-15T17:58:36.285479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:55:54.644188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.284044+0000","last_change":"2024-08-15T17:58:36.284044+0000","last_active":"2024-08-15T17:58:36.284044+0000","last_peered":"2024-08-15T17:58:36.284044+0000","last_clean":"2024-08-15T17:58:36.284044+0000","last_became_active":"2024-08-15T17:58:36.283664+0000","last_became_peered":"2024-08-15T17:58:36.283664+0000","last_unstale":"2024-08-15T17:58:36.284044+0000","last_undegraded":"2024-08-15T17:58:36.284044+0000","last_fullsized":"2024-08-15T17:58:36.284044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:58:52.618737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.287149+0000","last_change":"2024-08-15T17:58:36.287149+0000","last_active":"2024-08-15T17:58:36.287149+0000","last_peered":"2024-08-15T17:58:36.287149+0000","last_clean":"2024-08-15T17:58:36.287149+0000","last_became_active":"2024-08-15T17:58:36.286239+0000","last_became_peered":"2024-08-15T17:58:36.286239+0000","last_unstale":"2024-08-15T17:58:36.287149+0000","last_undegraded":"2024-08-15T17:58:36.287149+0000","last_fullsized":"2024-08-15T17:58:36.287149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:02:13.223698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283315+0000","last_change":"2024-08-15T17:58:36.283315+0000","last_active":"2024-08-15T17:58:36.283315+0000","last_peered":"2024-08-15T17:58:36.283315+0000","last_clean":"2024-08-15T17:58:36.283315+0000","last_became_active":"2024-08-15T17:58:36.282873+0000","last_became_peered":"2024-08-15T17:58:36.282873+0000","last_unstale":"2024-08-15T17:58:36.283315+0000","last_undegraded":"2024-08-15T17:58:36.283315+0000","last_fullsized":"2024-08-15T17:58:36.283315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:31:12.400383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.284677+0000","last_change":"2024-08-15T17:58:36.284677+0000","last_active":"2024-08-15T17:58:36.284677+0000","last_peered":"2024-08-15T17:58:36.284677+0000","last_clean":"2024-08-15T17:58:36.284677+0000","last_became_active":"2024-08-15T17:58:36.281734+0000","last_became_peered":"2024-08-15T17:58:36.281734+0000","last_unstale":"2024-08-15T17:58:36.284677+0000","last_undegraded":"2024-08-15T17:58:36.284677+0000","last_fullsized":"2024-08-15T17:58:36.284677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.281437+0000","last_change":"2024-08-15T17:58:36.281437+0000","last_active":"2024-08-15T17:58:36.281437+0000","last_peered":"2024-08-15T17:58:36.281437+0000","last_clean":"2024-08-15T17:58:36.281437+0000","last_became_active":"2024-08-15T17:58:36.280874+0000","last_became_peered":"2024-08-15T17:58:36.280874+0000","last_unstale":"2024-08-15T17:58:36.281437+0000","last_undegraded":"2024-08-15T17:58:36.281437+0000","last_fullsized":"2024-08-15T17:58:36.281437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:05.015208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285476+0000","last_change":"2024-08-15T17:58:36.285476+0000","last_active":"2024-08-15T17:58:36.285476+0000","last_peered":"2024-08-15T17:58:36.285476+0000","last_clean":"2024-08-15T17:58:36.285476+0000","last_became_active":"2024-08-15T17:58:36.285049+0000","last_became_peered":"2024-08-15T17:58:36.285049+0000","last_unstale":"2024-08-15T17:58:36.285476+0000","last_undegraded":"2024-08-15T17:58:36.285476+0000","last_fullsized":"2024-08-15T17:58:36.285476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:32:58.557897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280217+0000","last_change":"2024-08-15T17:58:35.277170+0000","last_active":"2024-08-15T17:58:36.280217+0000","last_peered":"2024-08-15T17:58:36.280217+0000","last_clean":"2024-08-15T17:58:36.280217+0000","last_became_active":"2024-08-15T17:58:35.276904+0000","last_became_peered":"2024-08-15T17:58:35.276904+0000","last_unstale":"2024-08-15T17:58:36.280217+0000","last_undegraded":"2024-08-15T17:58:36.280217+0000","last_fullsized":"2024-08-15T17:58:36.280217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:11:34.747972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.271912+0000","last_change":"2024-08-15T17:58:36.271912+0000","last_active":"2024-08-15T17:58:36.271912+0000","last_peered":"2024-08-15T17:58:36.271912+0000","last_clean":"2024-08-15T17:58:36.271912+0000","last_became_active":"2024-08-15T17:58:36.271346+0000","last_became_peered":"2024-08-15T17:58:36.271346+0000","last_unstale":"2024-08-15T17:58:36.271912+0000","last_undegraded":"2024-08-15T17:58:36.271912+0000","last_fullsized":"2024-08-15T17:58:36.271912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:54:02.548772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.280190+0000","last_change":"2024-08-15T17:58:35.275168+0000","last_active":"2024-08-15T17:58:35.280190+0000","last_peered":"2024-08-15T17:58:35.280190+0000","last_clean":"2024-08-15T17:58:35.280190+0000","last_became_active":"2024-08-15T17:58:35.274863+0000","last_became_peered":"2024-08-15T17:58:35.274863+0000","last_unstale":"2024-08-15T17:58:35.280190+0000","last_undegraded":"2024-08-15T17:58:35.280190+0000","last_fullsized":"2024-08-15T17:58:35.280190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:17:09.213636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290048+0000","last_change":"2024-08-15T17:58:36.290048+0000","last_active":"2024-08-15T17:58:36.290048+0000","last_peered":"2024-08-15T17:58:36.290048+0000","last_clean":"2024-08-15T17:58:36.290048+0000","last_became_active":"2024-08-15T17:58:36.289720+0000","last_became_peered":"2024-08-15T17:58:36.289720+0000","last_unstale":"2024-08-15T17:58:36.290048+0000","last_undegraded":"2024-08-15T17:58:36.290048+0000","last_fullsized":"2024-08-15T17:58:36.290048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:55.518837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.276294+0000","last_change":"2024-08-15T17:58:35.266688+0000","last_active":"2024-08-15T17:58:35.276294+0000","last_peered":"2024-08-15T17:58:35.276294+0000","last_clean":"2024-08-15T17:58:35.276294+0000","last_became_active":"2024-08-15T17:58:35.266306+0000","last_became_peered":"2024-08-15T17:58:35.266306+0000","last_unstale":"2024-08-15T17:58:35.276294+0000","last_undegraded":"2024-08-15T17:58:35.276294+0000","last_fullsized":"2024-08-15T17:58:35.276294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:11:34.747972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.287582+0000","last_change":"2024-08-15T17:58:36.287582+0000","last_active":"2024-08-15T17:58:36.287582+0000","last_peered":"2024-08-15T17:58:36.287582+0000","last_clean":"2024-08-15T17:58:36.287582+0000","last_became_active":"2024-08-15T17:58:36.286980+0000","last_became_peered":"2024-08-15T17:58:36.286980+0000","last_unstale":"2024-08-15T17:58:36.287582+0000","last_undegraded":"2024-08-15T17:58:36.287582+0000","last_fullsized":"2024-08-15T17:58:36.287582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:49:19.919158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280965+0000","last_change":"2024-08-15T17:58:36.280965+0000","last_active":"2024-08-15T17:58:36.280965+0000","last_peered":"2024-08-15T17:58:36.280965+0000","last_clean":"2024-08-15T17:58:36.280965+0000","last_became_active":"2024-08-15T17:58:36.280684+0000","last_became_peered":"2024-08-15T17:58:36.280684+0000","last_unstale":"2024-08-15T17:58:36.280965+0000","last_undegraded":"2024-08-15T17:58:36.280965+0000","last_fullsized":"2024-08-15T17:58:36.280965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:15:59.759500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.276504+0000","last_change":"2024-08-15T17:58:35.268261+0000","last_active":"2024-08-15T17:58:35.276504+0000","last_peered":"2024-08-15T17:58:35.276504+0000","last_clean":"2024-08-15T17:58:35.276504+0000","last_became_active":"2024-08-15T17:58:35.266109+0000","last_became_peered":"2024-08-15T17:58:35.266109+0000","last_unstale":"2024-08-15T17:58:35.276504+0000","last_undegraded":"2024-08-15T17:58:35.276504+0000","last_fullsized":"2024-08-15T17:58:35.276504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:06:34.826184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283639+0000","last_change":"2024-08-15T17:58:36.283639+0000","last_active":"2024-08-15T17:58:36.283639+0000","last_peered":"2024-08-15T17:58:36.283639+0000","last_clean":"2024-08-15T17:58:36.283639+0000","last_became_active":"2024-08-15T17:58:36.282343+0000","last_became_peered":"2024-08-15T17:58:36.282343+0000","last_unstale":"2024-08-15T17:58:36.283639+0000","last_undegraded":"2024-08-15T17:58:36.283639+0000","last_fullsized":"2024-08-15T17:58:36.283639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:58:52.618737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.300516+0000","last_change":"2024-08-15T17:58:35.276856+0000","last_active":"2024-08-15T17:58:36.300516+0000","last_peered":"2024-08-15T17:58:36.300516+0000","last_clean":"2024-08-15T17:58:36.300516+0000","last_became_active":"2024-08-15T17:58:35.275745+0000","last_became_peered":"2024-08-15T17:58:35.275745+0000","last_unstale":"2024-08-15T17:58:36.300516+0000","last_undegraded":"2024-08-15T17:58:36.300516+0000","last_fullsized":"2024-08-15T17:58:36.300516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:26:33.980860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.305509+0000","last_change":"2024-08-15T17:58:35.268853+0000","last_active":"2024-08-15T17:58:36.305509+0000","last_peered":"2024-08-15T17:58:36.305509+0000","last_clean":"2024-08-15T17:58:36.305509+0000","last_became_active":"2024-08-15T17:58:35.268541+0000","last_became_peered":"2024-08-15T17:58:35.268541+0000","last_unstale":"2024-08-15T17:58:36.305509+0000","last_undegraded":"2024-08-15T17:58:36.305509+0000","last_fullsized":"2024-08-15T17:58:36.305509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:33:31.329931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.278321+0000","last_change":"2024-08-15T17:58:36.278321+0000","last_active":"2024-08-15T17:58:36.278321+0000","last_peered":"2024-08-15T17:58:36.278321+0000","last_clean":"2024-08-15T17:58:36.278321+0000","last_became_active":"2024-08-15T17:58:36.278014+0000","last_became_peered":"2024-08-15T17:58:36.278014+0000","last_unstale":"2024-08-15T17:58:36.278321+0000","last_undegraded":"2024-08-15T17:58:36.278321+0000","last_fullsized":"2024-08-15T17:58:36.278321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:27:54.650775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.279183+0000","last_change":"2024-08-15T17:58:35.278113+0000","last_active":"2024-08-15T17:58:35.279183+0000","last_peered":"2024-08-15T17:58:35.279183+0000","last_clean":"2024-08-15T17:58:35.279183+0000","last_became_active":"2024-08-15T17:58:35.277811+0000","last_became_peered":"2024-08-15T17:58:35.277811+0000","last_unstale":"2024-08-15T17:58:35.279183+0000","last_undegraded":"2024-08-15T17:58:35.279183+0000","last_fullsized":"2024-08-15T17:58:35.279183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:12:40.711380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.304134+0000","last_change":"2024-08-15T17:58:35.271260+0000","last_active":"2024-08-15T17:58:36.304134+0000","last_peered":"2024-08-15T17:58:36.304134+0000","last_clean":"2024-08-15T17:58:36.304134+0000","last_became_active":"2024-08-15T17:58:35.270816+0000","last_became_peered":"2024-08-15T17:58:35.270816+0000","last_unstale":"2024-08-15T17:58:36.304134+0000","last_undegraded":"2024-08-15T17:58:36.304134+0000","last_fullsized":"2024-08-15T17:58:36.304134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:58:46.815165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.300580+0000","last_change":"2024-08-15T17:58:35.273755+0000","last_active":"2024-08-15T17:58:36.300580+0000","last_peered":"2024-08-15T17:58:36.300580+0000","last_clean":"2024-08-15T17:58:36.300580+0000","last_became_active":"2024-08-15T17:58:35.273318+0000","last_became_peered":"2024-08-15T17:58:35.273318+0000","last_unstale":"2024-08-15T17:58:36.300580+0000","last_undegraded":"2024-08-15T17:58:36.300580+0000","last_fullsized":"2024-08-15T17:58:36.300580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:13.047239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.273091+0000","last_change":"2024-08-15T17:58:35.274846+0000","last_active":"2024-08-15T17:58:36.273091+0000","last_peered":"2024-08-15T17:58:36.273091+0000","last_clean":"2024-08-15T17:58:36.273091+0000","last_became_active":"2024-08-15T17:58:35.274543+0000","last_became_peered":"2024-08-15T17:58:35.274543+0000","last_unstale":"2024-08-15T17:58:36.273091+0000","last_undegraded":"2024-08-15T17:58:36.273091+0000","last_fullsized":"2024-08-15T17:58:36.273091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:11:26.241336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285388+0000","last_change":"2024-08-15T17:58:36.285388+0000","last_active":"2024-08-15T17:58:36.285388+0000","last_peered":"2024-08-15T17:58:36.285388+0000","last_clean":"2024-08-15T17:58:36.285388+0000","last_became_active":"2024-08-15T17:58:36.285136+0000","last_became_peered":"2024-08-15T17:58:36.285136+0000","last_unstale":"2024-08-15T17:58:36.285388+0000","last_undegraded":"2024-08-15T17:58:36.285388+0000","last_fullsized":"2024-08-15T17:58:36.285388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:43.158602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.273848+0000","last_change":"2024-08-15T17:58:35.272166+0000","last_active":"2024-08-15T17:58:35.273848+0000","last_peered":"2024-08-15T17:58:35.273848+0000","last_clean":"2024-08-15T17:58:35.273848+0000","last_became_active":"2024-08-15T17:58:35.271841+0000","last_became_peered":"2024-08-15T17:58:35.271841+0000","last_unstale":"2024-08-15T17:58:35.273848+0000","last_undegraded":"2024-08-15T17:58:35.273848+0000","last_fullsized":"2024-08-15T17:58:35.273848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:40:09.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":159,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-08-15T17:58:26.107591+0000","last_change":"2024-08-15T17:58:24.394014+0000","last_active":"2024-08-15T17:58:26.107591+0000","last_peered":"2024-08-15T17:58:26.107591+0000","last_clean":"2024-08-15T17:58:26.107591+0000","last_became_active":"2024-08-15T17:58:24.391735+0000","last_became_peered":"2024-08-15T17:58:24.391735+0000","last_unstale":"2024-08-15T17:58:26.107591+0000","last_undegraded":"2024-08-15T17:58:26.107591+0000","last_fullsized":"2024-08-15T17:58:26.107591+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_clean_scrub_stamp":"2024-08-15T17:58:22.996335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:31.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.276469+0000","last_change":"2024-08-15T17:58:35.269765+0000","last_active":"2024-08-15T17:58:35.276469+0000","last_peered":"2024-08-15T17:58:35.276469+0000","last_clean":"2024-08-15T17:58:35.276469+0000","last_became_active":"2024-08-15T17:58:35.269249+0000","last_became_peered":"2024-08-15T17:58:35.269249+0000","last_unstale":"2024-08-15T17:58:35.276469+0000","last_undegraded":"2024-08-15T17:58:35.276469+0000","last_fullsized":"2024-08-15T17:58:35.276469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:27:34.522329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286437+0000","last_change":"2024-08-15T17:58:36.286437+0000","last_active":"2024-08-15T17:58:36.286437+0000","last_peered":"2024-08-15T17:58:36.286437+0000","last_clean":"2024-08-15T17:58:36.286437+0000","last_became_active":"2024-08-15T17:58:36.285909+0000","last_became_peered":"2024-08-15T17:58:36.285909+0000","last_unstale":"2024-08-15T17:58:36.286437+0000","last_undegraded":"2024-08-15T17:58:36.286437+0000","last_fullsized":"2024-08-15T17:58:36.286437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:47:17.486762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.277792+0000","last_change":"2024-08-15T17:58:36.277792+0000","last_active":"2024-08-15T17:58:36.277792+0000","last_peered":"2024-08-15T17:58:36.277792+0000","last_clean":"2024-08-15T17:58:36.277792+0000","last_became_active":"2024-08-15T17:58:36.276715+0000","last_became_peered":"2024-08-15T17:58:36.276715+0000","last_unstale":"2024-08-15T17:58:36.277792+0000","last_undegraded":"2024-08-15T17:58:36.277792+0000","last_fullsized":"2024-08-15T17:58:36.277792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:59:27.743235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.281618+0000","last_change":"2024-08-15T17:58:35.280834+0000","last_active":"2024-08-15T17:58:35.281618+0000","last_peered":"2024-08-15T17:58:35.281618+0000","last_clean":"2024-08-15T17:58:35.281618+0000","last_became_active":"2024-08-15T17:58:35.280524+0000","last_became_peered":"2024-08-15T17:58:35.280524+0000","last_unstale":"2024-08-15T17:58:35.281618+0000","last_undegraded":"2024-08-15T17:58:35.281618+0000","last_fullsized":"2024-08-15T17:58:35.281618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:00:50.031787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283772+0000","last_change":"2024-08-15T17:58:36.283772+0000","last_active":"2024-08-15T17:58:36.283772+0000","last_peered":"2024-08-15T17:58:36.283772+0000","last_clean":"2024-08-15T17:58:36.283772+0000","last_became_active":"2024-08-15T17:58:36.283482+0000","last_became_peered":"2024-08-15T17:58:36.283482+0000","last_unstale":"2024-08-15T17:58:36.283772+0000","last_undegraded":"2024-08-15T17:58:36.283772+0000","last_fullsized":"2024-08-15T17:58:36.283772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:33.556488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.276501+0000","last_change":"2024-08-15T17:58:35.270088+0000","last_active":"2024-08-15T17:58:35.276501+0000","last_peered":"2024-08-15T17:58:35.276501+0000","last_clean":"2024-08-15T17:58:35.276501+0000","last_became_active":"2024-08-15T17:58:35.268586+0000","last_became_peered":"2024-08-15T17:58:35.268586+0000","last_unstale":"2024-08-15T17:58:35.276501+0000","last_undegraded":"2024-08-15T17:58:35.276501+0000","last_fullsized":"2024-08-15T17:58:35.276501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:17:09.213636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.272193+0000","last_change":"2024-08-15T17:58:35.270974+0000","last_active":"2024-08-15T17:58:35.272193+0000","last_peered":"2024-08-15T17:58:35.272193+0000","last_clean":"2024-08-15T17:58:35.272193+0000","last_became_active":"2024-08-15T17:58:35.270645+0000","last_became_peered":"2024-08-15T17:58:35.270645+0000","last_unstale":"2024-08-15T17:58:35.272193+0000","last_undegraded":"2024-08-15T17:58:35.272193+0000","last_fullsized":"2024-08-15T17:58:35.272193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:44.174826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.282072+0000","last_change":"2024-08-15T17:58:36.282072+0000","last_active":"2024-08-15T17:58:36.282072+0000","last_peered":"2024-08-15T17:58:36.282072+0000","last_clean":"2024-08-15T17:58:36.282072+0000","last_became_active":"2024-08-15T17:58:36.281850+0000","last_became_peered":"2024-08-15T17:58:36.281850+0000","last_unstale":"2024-08-15T17:58:36.282072+0000","last_undegraded":"2024-08-15T17:58:36.282072+0000","last_fullsized":"2024-08-15T17:58:36.282072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:54:56.996071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.284264+0000","last_change":"2024-08-15T17:58:36.284264+0000","last_active":"2024-08-15T17:58:36.284264+0000","last_peered":"2024-08-15T17:58:36.284264+0000","last_clean":"2024-08-15T17:58:36.284264+0000","last_became_active":"2024-08-15T17:58:36.283964+0000","last_became_peered":"2024-08-15T17:58:36.283964+0000","last_unstale":"2024-08-15T17:58:36.284264+0000","last_undegraded":"2024-08-15T17:58:36.284264+0000","last_fullsized":"2024-08-15T17:58:36.284264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:07:03.605391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.277468+0000","last_change":"2024-08-15T17:58:35.274546+0000","last_active":"2024-08-15T17:58:35.277468+0000","last_peered":"2024-08-15T17:58:35.277468+0000","last_clean":"2024-08-15T17:58:35.277468+0000","last_became_active":"2024-08-15T17:58:35.274232+0000","last_became_peered":"2024-08-15T17:58:35.274232+0000","last_unstale":"2024-08-15T17:58:35.277468+0000","last_undegraded":"2024-08-15T17:58:35.277468+0000","last_fullsized":"2024-08-15T17:58:35.277468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:17.590038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290821+0000","last_change":"2024-08-15T17:58:36.290821+0000","last_active":"2024-08-15T17:58:36.290821+0000","last_peered":"2024-08-15T17:58:36.290821+0000","last_clean":"2024-08-15T17:58:36.290821+0000","last_became_active":"2024-08-15T17:58:36.290495+0000","last_became_peered":"2024-08-15T17:58:36.290495+0000","last_unstale":"2024-08-15T17:58:36.290821+0000","last_undegraded":"2024-08-15T17:58:36.290821+0000","last_fullsized":"2024-08-15T17:58:36.290821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:43.158602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.284750+0000","last_change":"2024-08-15T17:58:36.284750+0000","last_active":"2024-08-15T17:58:36.284750+0000","last_peered":"2024-08-15T17:58:36.284750+0000","last_clean":"2024-08-15T17:58:36.284750+0000","last_became_active":"2024-08-15T17:58:36.284480+0000","last_became_peered":"2024-08-15T17:58:36.284480+0000","last_unstale":"2024-08-15T17:58:36.284750+0000","last_undegraded":"2024-08-15T17:58:36.284750+0000","last_fullsized":"2024-08-15T17:58:36.284750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:55.518837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.278361+0000","last_change":"2024-08-15T17:58:35.277768+0000","last_active":"2024-08-15T17:58:35.278361+0000","last_peered":"2024-08-15T17:58:35.278361+0000","last_clean":"2024-08-15T17:58:35.278361+0000","last_became_active":"2024-08-15T17:58:35.277217+0000","last_became_peered":"2024-08-15T17:58:35.277217+0000","last_unstale":"2024-08-15T17:58:35.278361+0000","last_undegraded":"2024-08-15T17:58:35.278361+0000","last_fullsized":"2024-08-15T17:58:35.278361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:13.047239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289433+0000","last_change":"2024-08-15T17:58:36.289433+0000","last_active":"2024-08-15T17:58:36.289433+0000","last_peered":"2024-08-15T17:58:36.289433+0000","last_clean":"2024-08-15T17:58:36.289433+0000","last_became_active":"2024-08-15T17:58:36.288625+0000","last_became_peered":"2024-08-15T17:58:36.288625+0000","last_unstale":"2024-08-15T17:58:36.289433+0000","last_undegraded":"2024-08-15T17:58:36.289433+0000","last_fullsized":"2024-08-15T17:58:36.289433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:47:03.102185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.278220+0000","last_change":"2024-08-15T17:58:35.276117+0000","last_active":"2024-08-15T17:58:35.278220+0000","last_peered":"2024-08-15T17:58:35.278220+0000","last_clean":"2024-08-15T17:58:35.278220+0000","last_became_active":"2024-08-15T17:58:35.275707+0000","last_became_peered":"2024-08-15T17:58:35.275707+0000","last_unstale":"2024-08-15T17:58:35.278220+0000","last_undegraded":"2024-08-15T17:58:35.278220+0000","last_fullsized":"2024-08-15T17:58:35.278220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:40:09.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290279+0000","last_change":"2024-08-15T17:58:36.290279+0000","last_active":"2024-08-15T17:58:36.290279+0000","last_peered":"2024-08-15T17:58:36.290279+0000","last_clean":"2024-08-15T17:58:36.290279+0000","last_became_active":"2024-08-15T17:58:36.289946+0000","last_became_peered":"2024-08-15T17:58:36.289946+0000","last_unstale":"2024-08-15T17:58:36.290279+0000","last_undegraded":"2024-08-15T17:58:36.290279+0000","last_fullsized":"2024-08-15T17:58:36.290279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:35.242412+0000","last_change":"2024-08-15T17:58:35.242412+0000","last_active":"2024-08-15T17:58:35.242412+0000","last_peered":"2024-08-15T17:58:35.242412+0000","last_clean":"2024-08-15T17:58:35.242412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:35.242412+0000","last_undegraded":"2024-08-15T17:58:35.242412+0000","last_fullsized":"2024-08-15T17:58:35.242412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.271574+0000","last_change":"2024-08-15T17:58:36.271574+0000","last_active":"2024-08-15T17:58:36.271574+0000","last_peered":"2024-08-15T17:58:36.271574+0000","last_clean":"2024-08-15T17:58:36.271574+0000","last_became_active":"2024-08-15T17:58:36.271259+0000","last_became_peered":"2024-08-15T17:58:36.271259+0000","last_unstale":"2024-08-15T17:58:36.271574+0000","last_undegraded":"2024-08-15T17:58:36.271574+0000","last_fullsized":"2024-08-15T17:58:36.271574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.281039+0000","last_change":"2024-08-15T17:58:36.281039+0000","last_active":"2024-08-15T17:58:36.281039+0000","last_peered":"2024-08-15T17:58:36.281039+0000","last_clean":"2024-08-15T17:58:36.281039+0000","last_became_active":"2024-08-15T17:58:36.280780+0000","last_became_peered":"2024-08-15T17:58:36.280780+0000","last_unstale":"2024-08-15T17:58:36.281039+0000","last_undegraded":"2024-08-15T17:58:36.281039+0000","last_fullsized":"2024-08-15T17:58:36.281039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:47:03.102185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:58:34.233433+0000","last_change":"2024-08-15T17:58:34.233433+0000","last_active":"2024-08-15T17:58:34.233433+0000","last_peered":"2024-08-15T17:58:34.233433+0000","last_clean":"2024-08-15T17:58:34.233433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:58:34.233433+0000","last_undegraded":"2024-08-15T17:58:34.233433+0000","last_fullsized":"2024-08-15T17:58:34.233433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.282308+0000","last_change":"2024-08-15T17:58:36.282308+0000","last_active":"2024-08-15T17:58:36.282308+0000","last_peered":"2024-08-15T17:58:36.282308+0000","last_clean":"2024-08-15T17:58:36.282308+0000","last_became_active":"2024-08-15T17:58:36.281986+0000","last_became_peered":"2024-08-15T17:58:36.281986+0000","last_unstale":"2024-08-15T17:58:36.282308+0000","last_undegraded":"2024-08-15T17:58:36.282308+0000","last_fullsized":"2024-08-15T17:58:36.282308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:48:28.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280663+0000","last_change":"2024-08-15T17:58:35.274614+0000","last_active":"2024-08-15T17:58:36.280663+0000","last_peered":"2024-08-15T17:58:36.280663+0000","last_clean":"2024-08-15T17:58:36.280663+0000","last_became_active":"2024-08-15T17:58:35.274185+0000","last_became_peered":"2024-08-15T17:58:35.274185+0000","last_unstale":"2024-08-15T17:58:36.280663+0000","last_undegraded":"2024-08-15T17:58:36.280663+0000","last_fullsized":"2024-08-15T17:58:36.280663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:41.642352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290572+0000","last_change":"2024-08-15T17:58:36.290572+0000","last_active":"2024-08-15T17:58:36.290572+0000","last_peered":"2024-08-15T17:58:36.290572+0000","last_clean":"2024-08-15T17:58:36.290572+0000","last_became_active":"2024-08-15T17:58:36.290269+0000","last_became_peered":"2024-08-15T17:58:36.290269+0000","last_unstale":"2024-08-15T17:58:36.290572+0000","last_undegraded":"2024-08-15T17:58:36.290572+0000","last_fullsized":"2024-08-15T17:58:36.290572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:56:49.069407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301913+0000","last_change":"2024-08-15T17:58:35.276392+0000","last_active":"2024-08-15T17:58:36.301913+0000","last_peered":"2024-08-15T17:58:36.301913+0000","last_clean":"2024-08-15T17:58:36.301913+0000","last_became_active":"2024-08-15T17:58:35.276085+0000","last_became_peered":"2024-08-15T17:58:35.276085+0000","last_unstale":"2024-08-15T17:58:36.301913+0000","last_undegraded":"2024-08-15T17:58:36.301913+0000","last_fullsized":"2024-08-15T17:58:36.301913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:10:36.097809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":82,"acting":82,"num_store_stats":5},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":40,"acting":40,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298716,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073124,"statfs":{"total":96636764160,"available":96330878976,"internally_reserved":0,"allocated":753664,"data_stored":638050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297516,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":159744,"data_stored":47682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297512,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":159744,"data_stored":47682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":200704,"data_stored":66603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":208896,"data_stored":68065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301700,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070140,"statfs":{"total":96636764160,"available":96327823360,"internally_reserved":0,"allocated":184320,"data_stored":59750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301724,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070116,"statfs":{"total":96636764160,"available":96327798784,"internally_reserved":0,"allocated":204800,"data_stored":66673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":778240,"data_stored":650118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:58:39.897 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:58:40.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.006+0000 7eff414f7700 1 Processor -- start 2024-08-15T17:58:40.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.006+0000 7eff414f7700 1 -- start start 2024-08-15T17:58:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff3c142b50 0x7eff3c0ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7eff3c0cae50 0x7eff3c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3c0ca510 con 0x7eff3c142780 2024-08-15T17:58:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3c0cb290 con 0x7eff3c0cae50 2024-08-15T17:58:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0c9900 con 0x7eff3c142b50 2024-08-15T17:58:40.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff40cf6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7eff3c0cae50 0x7eff3c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff3bfff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7eff3c142780 0x7eff3c0ae9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40076/0 (socket says 172.21.15.38:40076) 2024-08-15T17:58:40.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff3bfff700 1 -- 172.21.15.38:0/3848393716 learned_addr learned my addr 172.21.15.38:0/3848393716 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:40.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff3b7fe700 1 --2- 172.21.15.38:0/3848393716 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff3c142b50 0x7eff3c0ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff40cf6700 1 -- 172.21.15.38:0/3848393716 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff3c142b50 msgr2=0x7eff3c0ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff40cf6700 1 --2- 172.21.15.38:0/3848393716 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff3c142b50 0x7eff3c0ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff40cf6700 1 -- 172.21.15.38:0/3848393716 >> v1:172.21.15.38:6789/0 conn(0x7eff3c142780 legacy=0x7eff3c0ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff3affd700 1 -- 172.21.15.38:0/3848393716 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274344104 0 0) 0x7eff3c0ca510 con 0x7eff3c142780 2024-08-15T17:58:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff40cf6700 1 -- 172.21.15.38:0/3848393716 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c0c9be0 con 0x7eff3c0cae50 2024-08-15T17:58:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff40cf6700 1 --2- 172.21.15.38:0/3848393716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3c0cae50 0x7eff3c15a930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff34009e60 tx=0x7eff34009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c018584bfcaa05 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff3affd700 1 -- 172.21.15.38:0/3848393716 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3400e070 con 0x7eff3c0cae50 2024-08-15T17:58:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff3affd700 1 -- 172.21.15.38:0/3848393716 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3400bb60 con 0x7eff3c0cae50 2024-08-15T17:58:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff3affd700 1 -- 172.21.15.38:0/3848393716 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34012430 con 0x7eff3c0cae50 2024-08-15T17:58:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 -- 172.21.15.38:0/3848393716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3c0cae50 msgr2=0x7eff3c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 --2- 172.21.15.38:0/3848393716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3c0cae50 0x7eff3c15a930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff34009e60 tx=0x7eff34009780 comp rx=0 tx=0).stop 2024-08-15T17:58:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 -- 172.21.15.38:0/3848393716 shutdown_connections 2024-08-15T17:58:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 --2- 172.21.15.38:0/3848393716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3c0cae50 0x7eff3c15a930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 --2- 172.21.15.38:0/3848393716 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff3c142b50 0x7eff3c0ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 -- 172.21.15.38:0/3848393716 >> 172.21.15.38:0/3848393716 conn(0x7eff3c06e070 msgr2=0x7eff3c0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 -- 172.21.15.38:0/3848393716 shutdown_connections 2024-08-15T17:58:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.010+0000 7eff414f7700 1 -- 172.21.15.38:0/3848393716 wait complete. 2024-08-15T17:58:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff414f7700 1 Processor -- start 2024-08-15T17:58:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff414f7700 1 -- start start 2024-08-15T17:58:40.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff414f7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff3c0cae50 0x7eff3c158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff414f7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3c142780 0x7eff3c13c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff414f7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c142b50 0x7eff3c139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff414f7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c13bec0 con 0x7eff3c142780 2024-08-15T17:58:40.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff414f7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c13b5d0 con 0x7eff3c142b50 2024-08-15T17:58:40.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff414f7700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c13b870 con 0x7eff3c0cae50 2024-08-15T17:58:40.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff3b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3c142780 0x7eff3c13c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff3b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3c142780 0x7eff3c13c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53590/0 (socket says 172.21.15.38:53590) 2024-08-15T17:58:40.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff3b7fe700 1 -- 172.21.15.38:0/2958032817 learned_addr learned my addr 172.21.15.38:0/2958032817 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff40cf6700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c142b50 0x7eff3c139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff3bfff700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff3c0cae50 0x7eff3c158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff40cf6700 1 -- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff3c0cae50 msgr2=0x7eff3c158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff40cf6700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff3c0cae50 0x7eff3c158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff40cf6700 1 -- 172.21.15.38:0/2958032817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3c142780 msgr2=0x7eff3c13c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff40cf6700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3c142780 0x7eff3c13c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff40cf6700 1 -- 172.21.15.38:0/2958032817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff34009a20 con 0x7eff3c142b50 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff3b7fe700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3c142780 0x7eff3c13c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff40cf6700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c142b50 0x7eff3c139390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff34016000 tx=0x7eff34019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff33fff700 1 -- 172.21.15.38:0/2958032817 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34012ea0 con 0x7eff3c142b50 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff33fff700 1 -- 172.21.15.38:0/2958032817 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff34027470 con 0x7eff3c142b50 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff414f7700 1 -- 172.21.15.38:0/2958032817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c13bb50 con 0x7eff3c142b50 2024-08-15T17:58:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff33fff700 1 -- 172.21.15.38:0/2958032817 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34027830 con 0x7eff3c142b50 2024-08-15T17:58:40.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.014+0000 7eff414f7700 1 -- 172.21.15.38:0/2958032817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c16a000 con 0x7eff3c142b50 2024-08-15T17:58:40.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.018+0000 7eff414f7700 1 -- 172.21.15.38:0/2958032817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c13c030 con 0x7eff3c142b50 2024-08-15T17:58:40.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.018+0000 7eff33fff700 1 -- 172.21.15.38:0/2958032817 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7eff34024070 con 0x7eff3c142b50 2024-08-15T17:58:40.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.018+0000 7eff33fff700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff1c060860 0x7eff1c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.018+0000 7eff33fff700 1 -- 172.21.15.38:0/2958032817 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7eff3407f1c0 con 0x7eff3c142b50 2024-08-15T17:58:40.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.018+0000 7eff3bfff700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff1c060860 0x7eff1c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.018+0000 7eff3bfff700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff1c060860 0x7eff1c062d10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7eff2c007fd0 tx=0x7eff2c007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:40.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.030+0000 7eff33fff700 1 -- 172.21.15.38:0/2958032817 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7eff3404f610 con 0x7eff3c142b50 2024-08-15T17:58:40.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.190+0000 7eff414f7700 1 -- 172.21.15.38:0/2958032817 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff3c0638f0 con 0x7eff1c060860 2024-08-15T17:58:40.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.206+0000 7eff33fff700 1 -- 172.21.15.38:0/2958032817 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264427 (secure 0 0 0) 0x7eff3c0638f0 con 0x7eff1c060860 2024-08-15T17:58:40.213 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:40.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.210+0000 7eff414f7700 1 -- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff1c060860 msgr2=0x7eff1c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.210+0000 7eff414f7700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff1c060860 0x7eff1c062d10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7eff2c007fd0 tx=0x7eff2c007d30 comp rx=0 tx=0).stop 2024-08-15T17:58:40.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.210+0000 7eff414f7700 1 -- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c142b50 msgr2=0x7eff3c139390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.210+0000 7eff414f7700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c142b50 0x7eff3c139390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff34016000 tx=0x7eff34019000 comp rx=0 tx=0).stop 2024-08-15T17:58:40.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.210+0000 7eff414f7700 1 -- 172.21.15.38:0/2958032817 shutdown_connections 2024-08-15T17:58:40.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.210+0000 7eff414f7700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7eff1c060860 0x7eff1c062d10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.210+0000 7eff414f7700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c142b50 0x7eff3c139390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.210+0000 7eff414f7700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3c142780 0x7eff3c13c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.210+0000 7eff414f7700 1 --2- 172.21.15.38:0/2958032817 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff3c0cae50 0x7eff3c158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.210+0000 7eff414f7700 1 -- 172.21.15.38:0/2958032817 >> 172.21.15.38:0/2958032817 conn(0x7eff3c06e070 msgr2=0x7eff3c15be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:40.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.210+0000 7eff414f7700 1 -- 172.21.15.38:0/2958032817 shutdown_connections 2024-08-15T17:58:40.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.214+0000 7eff414f7700 1 -- 172.21.15.38:0/2958032817 wait complete. 2024-08-15T17:58:40.216 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-08-15T17:58:40.229 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-08-15T17:58:40.168884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2389912,"kb_used_data":2724,"kb_used_omap":13,"kb_used_meta":2387058,"kb_avail":752584808,"statfs":{"total":773094113280,"available":770646843392,"internally_reserved":0,"allocated":2789376,"data_stored":1703364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2444348096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.927465"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.292335+0000","last_change":"2024-08-15T17:58:36.292335+0000","last_active":"2024-08-15T17:58:36.292335+0000","last_peered":"2024-08-15T17:58:36.292335+0000","last_clean":"2024-08-15T17:58:36.292335+0000","last_became_active":"2024-08-15T17:58:36.292033+0000","last_became_peered":"2024-08-15T17:58:36.292033+0000","last_unstale":"2024-08-15T17:58:36.292335+0000","last_undegraded":"2024-08-15T17:58:36.292335+0000","last_fullsized":"2024-08-15T17:58:36.292335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:59:24.721036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286540+0000","last_change":"2024-08-15T17:58:36.286540+0000","last_active":"2024-08-15T17:58:36.286540+0000","last_peered":"2024-08-15T17:58:36.286540+0000","last_clean":"2024-08-15T17:58:36.286540+0000","last_became_active":"2024-08-15T17:58:36.285657+0000","last_became_peered":"2024-08-15T17:58:36.285657+0000","last_unstale":"2024-08-15T17:58:36.286540+0000","last_undegraded":"2024-08-15T17:58:36.286540+0000","last_fullsized":"2024-08-15T17:58:36.286540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:50:39.313237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.279817+0000","last_change":"2024-08-15T17:58:36.279817+0000","last_active":"2024-08-15T17:58:36.279817+0000","last_peered":"2024-08-15T17:58:36.279817+0000","last_clean":"2024-08-15T17:58:36.279817+0000","last_became_active":"2024-08-15T17:58:36.279494+0000","last_became_peered":"2024-08-15T17:58:36.279494+0000","last_unstale":"2024-08-15T17:58:36.279817+0000","last_undegraded":"2024-08-15T17:58:36.279817+0000","last_fullsized":"2024-08-15T17:58:36.279817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:48:29.832597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285873+0000","last_change":"2024-08-15T17:58:36.285873+0000","last_active":"2024-08-15T17:58:36.285873+0000","last_peered":"2024-08-15T17:58:36.285873+0000","last_clean":"2024-08-15T17:58:36.285873+0000","last_became_active":"2024-08-15T17:58:36.285630+0000","last_became_peered":"2024-08-15T17:58:36.285630+0000","last_unstale":"2024-08-15T17:58:36.285873+0000","last_undegraded":"2024-08-15T17:58:36.285873+0000","last_fullsized":"2024-08-15T17:58:36.285873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:30:51.740686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.281869+0000","last_change":"2024-08-15T17:58:36.281869+0000","last_active":"2024-08-15T17:58:36.281869+0000","last_peered":"2024-08-15T17:58:36.281869+0000","last_clean":"2024-08-15T17:58:36.281869+0000","last_became_active":"2024-08-15T17:58:36.281550+0000","last_became_peered":"2024-08-15T17:58:36.281550+0000","last_unstale":"2024-08-15T17:58:36.281869+0000","last_undegraded":"2024-08-15T17:58:36.281869+0000","last_fullsized":"2024-08-15T17:58:36.281869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:33:16.117022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286578+0000","last_change":"2024-08-15T17:58:36.286578+0000","last_active":"2024-08-15T17:58:36.286578+0000","last_peered":"2024-08-15T17:58:36.286578+0000","last_clean":"2024-08-15T17:58:36.286578+0000","last_became_active":"2024-08-15T17:58:36.286298+0000","last_became_peered":"2024-08-15T17:58:36.286298+0000","last_unstale":"2024-08-15T17:58:36.286578+0000","last_undegraded":"2024-08-15T17:58:36.286578+0000","last_fullsized":"2024-08-15T17:58:36.286578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:47:17.486762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286906+0000","last_change":"2024-08-15T17:58:36.286906+0000","last_active":"2024-08-15T17:58:36.286906+0000","last_peered":"2024-08-15T17:58:36.286906+0000","last_clean":"2024-08-15T17:58:36.286906+0000","last_became_active":"2024-08-15T17:58:36.286620+0000","last_became_peered":"2024-08-15T17:58:36.286620+0000","last_unstale":"2024-08-15T17:58:36.286906+0000","last_undegraded":"2024-08-15T17:58:36.286906+0000","last_fullsized":"2024-08-15T17:58:36.286906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:35:39.618955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285830+0000","last_change":"2024-08-15T17:58:36.285830+0000","last_active":"2024-08-15T17:58:36.285830+0000","last_peered":"2024-08-15T17:58:36.285830+0000","last_clean":"2024-08-15T17:58:36.285830+0000","last_became_active":"2024-08-15T17:58:36.285422+0000","last_became_peered":"2024-08-15T17:58:36.285422+0000","last_unstale":"2024-08-15T17:58:36.285830+0000","last_undegraded":"2024-08-15T17:58:36.285830+0000","last_fullsized":"2024-08-15T17:58:36.285830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:15:59.759500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286462+0000","last_change":"2024-08-15T17:58:36.286462+0000","last_active":"2024-08-15T17:58:36.286462+0000","last_peered":"2024-08-15T17:58:36.286462+0000","last_clean":"2024-08-15T17:58:36.286462+0000","last_became_active":"2024-08-15T17:58:36.285671+0000","last_became_peered":"2024-08-15T17:58:36.285671+0000","last_unstale":"2024-08-15T17:58:36.286462+0000","last_undegraded":"2024-08-15T17:58:36.286462+0000","last_fullsized":"2024-08-15T17:58:36.286462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:22:51.492443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289845+0000","last_change":"2024-08-15T17:58:36.289845+0000","last_active":"2024-08-15T17:58:36.289845+0000","last_peered":"2024-08-15T17:58:36.289845+0000","last_clean":"2024-08-15T17:58:36.289845+0000","last_became_active":"2024-08-15T17:58:36.286963+0000","last_became_peered":"2024-08-15T17:58:36.286963+0000","last_unstale":"2024-08-15T17:58:36.289845+0000","last_undegraded":"2024-08-15T17:58:36.289845+0000","last_fullsized":"2024-08-15T17:58:36.289845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:49:19.919158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.292046+0000","last_change":"2024-08-15T17:58:36.292046+0000","last_active":"2024-08-15T17:58:36.292046+0000","last_peered":"2024-08-15T17:58:36.292046+0000","last_clean":"2024-08-15T17:58:36.292046+0000","last_became_active":"2024-08-15T17:58:36.291772+0000","last_became_peered":"2024-08-15T17:58:36.291772+0000","last_unstale":"2024-08-15T17:58:36.292046+0000","last_undegraded":"2024-08-15T17:58:36.292046+0000","last_fullsized":"2024-08-15T17:58:36.292046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:11:45.354221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.291455+0000","last_change":"2024-08-15T17:58:36.288242+0000","last_active":"2024-08-15T17:58:36.291455+0000","last_peered":"2024-08-15T17:58:36.291455+0000","last_clean":"2024-08-15T17:58:36.291455+0000","last_became_active":"2024-08-15T17:58:36.287991+0000","last_became_peered":"2024-08-15T17:58:36.287991+0000","last_unstale":"2024-08-15T17:58:36.291455+0000","last_undegraded":"2024-08-15T17:58:36.291455+0000","last_fullsized":"2024-08-15T17:58:36.291455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:11:27.250315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286478+0000","last_change":"2024-08-15T17:58:36.286478+0000","last_active":"2024-08-15T17:58:36.286478+0000","last_peered":"2024-08-15T17:58:36.286478+0000","last_clean":"2024-08-15T17:58:36.286478+0000","last_became_active":"2024-08-15T17:58:36.285967+0000","last_became_peered":"2024-08-15T17:58:36.285967+0000","last_unstale":"2024-08-15T17:58:36.286478+0000","last_undegraded":"2024-08-15T17:58:36.286478+0000","last_fullsized":"2024-08-15T17:58:36.286478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.279946+0000","last_change":"2024-08-15T17:58:36.279946+0000","last_active":"2024-08-15T17:58:36.279946+0000","last_peered":"2024-08-15T17:58:36.279946+0000","last_clean":"2024-08-15T17:58:36.279946+0000","last_became_active":"2024-08-15T17:58:36.279648+0000","last_became_peered":"2024-08-15T17:58:36.279648+0000","last_unstale":"2024-08-15T17:58:36.279946+0000","last_undegraded":"2024-08-15T17:58:36.279946+0000","last_fullsized":"2024-08-15T17:58:36.279946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:34:14.015831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286890+0000","last_change":"2024-08-15T17:58:36.286890+0000","last_active":"2024-08-15T17:58:36.286890+0000","last_peered":"2024-08-15T17:58:36.286890+0000","last_clean":"2024-08-15T17:58:36.286890+0000","last_became_active":"2024-08-15T17:58:36.286332+0000","last_became_peered":"2024-08-15T17:58:36.286332+0000","last_unstale":"2024-08-15T17:58:36.286890+0000","last_undegraded":"2024-08-15T17:58:36.286890+0000","last_fullsized":"2024-08-15T17:58:36.286890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:07:28.572500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.287994+0000","last_change":"2024-08-15T17:58:36.287994+0000","last_active":"2024-08-15T17:58:36.287994+0000","last_peered":"2024-08-15T17:58:36.287994+0000","last_clean":"2024-08-15T17:58:36.287994+0000","last_became_active":"2024-08-15T17:58:36.286645+0000","last_became_peered":"2024-08-15T17:58:36.286645+0000","last_unstale":"2024-08-15T17:58:36.287994+0000","last_undegraded":"2024-08-15T17:58:36.287994+0000","last_fullsized":"2024-08-15T17:58:36.287994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280114+0000","last_change":"2024-08-15T17:58:36.280114+0000","last_active":"2024-08-15T17:58:36.280114+0000","last_peered":"2024-08-15T17:58:36.280114+0000","last_clean":"2024-08-15T17:58:36.280114+0000","last_became_active":"2024-08-15T17:58:36.279785+0000","last_became_peered":"2024-08-15T17:58:36.279785+0000","last_unstale":"2024-08-15T17:58:36.280114+0000","last_undegraded":"2024-08-15T17:58:36.280114+0000","last_fullsized":"2024-08-15T17:58:36.280114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:56:03.150852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290521+0000","last_change":"2024-08-15T17:58:36.290521+0000","last_active":"2024-08-15T17:58:36.290521+0000","last_peered":"2024-08-15T17:58:36.290521+0000","last_clean":"2024-08-15T17:58:36.290521+0000","last_became_active":"2024-08-15T17:58:36.290154+0000","last_became_peered":"2024-08-15T17:58:36.290154+0000","last_unstale":"2024-08-15T17:58:36.290521+0000","last_undegraded":"2024-08-15T17:58:36.290521+0000","last_fullsized":"2024-08-15T17:58:36.290521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:45.183805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285210+0000","last_change":"2024-08-15T17:58:36.285210+0000","last_active":"2024-08-15T17:58:36.285210+0000","last_peered":"2024-08-15T17:58:36.285210+0000","last_clean":"2024-08-15T17:58:36.285210+0000","last_became_active":"2024-08-15T17:58:36.284752+0000","last_became_peered":"2024-08-15T17:58:36.284752+0000","last_unstale":"2024-08-15T17:58:36.285210+0000","last_undegraded":"2024-08-15T17:58:36.285210+0000","last_fullsized":"2024-08-15T17:58:36.285210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:18:07.754382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285554+0000","last_change":"2024-08-15T17:58:36.285554+0000","last_active":"2024-08-15T17:58:36.285554+0000","last_peered":"2024-08-15T17:58:36.285554+0000","last_clean":"2024-08-15T17:58:36.285554+0000","last_became_active":"2024-08-15T17:58:36.285281+0000","last_became_peered":"2024-08-15T17:58:36.285281+0000","last_unstale":"2024-08-15T17:58:36.285554+0000","last_undegraded":"2024-08-15T17:58:36.285554+0000","last_fullsized":"2024-08-15T17:58:36.285554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:12:39.229320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280839+0000","last_change":"2024-08-15T17:58:36.280839+0000","last_active":"2024-08-15T17:58:36.280839+0000","last_peered":"2024-08-15T17:58:36.280839+0000","last_clean":"2024-08-15T17:58:36.280839+0000","last_became_active":"2024-08-15T17:58:36.280519+0000","last_became_peered":"2024-08-15T17:58:36.280519+0000","last_unstale":"2024-08-15T17:58:36.280839+0000","last_undegraded":"2024-08-15T17:58:36.280839+0000","last_fullsized":"2024-08-15T17:58:36.280839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.278583+0000","last_change":"2024-08-15T17:58:36.278583+0000","last_active":"2024-08-15T17:58:36.278583+0000","last_peered":"2024-08-15T17:58:36.278583+0000","last_clean":"2024-08-15T17:58:36.278583+0000","last_became_active":"2024-08-15T17:58:36.278308+0000","last_became_peered":"2024-08-15T17:58:36.278308+0000","last_unstale":"2024-08-15T17:58:36.278583+0000","last_undegraded":"2024-08-15T17:58:36.278583+0000","last_fullsized":"2024-08-15T17:58:36.278583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:56:49.069407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283003+0000","last_change":"2024-08-15T17:58:36.283003+0000","last_active":"2024-08-15T17:58:36.283003+0000","last_peered":"2024-08-15T17:58:36.283003+0000","last_clean":"2024-08-15T17:58:36.283003+0000","last_became_active":"2024-08-15T17:58:36.282691+0000","last_became_peered":"2024-08-15T17:58:36.282691+0000","last_unstale":"2024-08-15T17:58:36.283003+0000","last_undegraded":"2024-08-15T17:58:36.283003+0000","last_fullsized":"2024-08-15T17:58:36.283003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:48:28.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285479+0000","last_change":"2024-08-15T17:58:36.285479+0000","last_active":"2024-08-15T17:58:36.285479+0000","last_peered":"2024-08-15T17:58:36.285479+0000","last_clean":"2024-08-15T17:58:36.285479+0000","last_became_active":"2024-08-15T17:58:36.285044+0000","last_became_peered":"2024-08-15T17:58:36.285044+0000","last_unstale":"2024-08-15T17:58:36.285479+0000","last_undegraded":"2024-08-15T17:58:36.285479+0000","last_fullsized":"2024-08-15T17:58:36.285479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:55:54.644188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.284044+0000","last_change":"2024-08-15T17:58:36.284044+0000","last_active":"2024-08-15T17:58:36.284044+0000","last_peered":"2024-08-15T17:58:36.284044+0000","last_clean":"2024-08-15T17:58:36.284044+0000","last_became_active":"2024-08-15T17:58:36.283664+0000","last_became_peered":"2024-08-15T17:58:36.283664+0000","last_unstale":"2024-08-15T17:58:36.284044+0000","last_undegraded":"2024-08-15T17:58:36.284044+0000","last_fullsized":"2024-08-15T17:58:36.284044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:58:52.618737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290152+0000","last_change":"2024-08-15T17:58:36.290152+0000","last_active":"2024-08-15T17:58:36.290152+0000","last_peered":"2024-08-15T17:58:36.290152+0000","last_clean":"2024-08-15T17:58:36.290152+0000","last_became_active":"2024-08-15T17:58:36.287149+0000","last_became_peered":"2024-08-15T17:58:36.287149+0000","last_unstale":"2024-08-15T17:58:36.290152+0000","last_undegraded":"2024-08-15T17:58:36.290152+0000","last_fullsized":"2024-08-15T17:58:36.290152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:50:39.313237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.287149+0000","last_change":"2024-08-15T17:58:36.287149+0000","last_active":"2024-08-15T17:58:36.287149+0000","last_peered":"2024-08-15T17:58:36.287149+0000","last_clean":"2024-08-15T17:58:36.287149+0000","last_became_active":"2024-08-15T17:58:36.286239+0000","last_became_peered":"2024-08-15T17:58:36.286239+0000","last_unstale":"2024-08-15T17:58:36.287149+0000","last_undegraded":"2024-08-15T17:58:36.287149+0000","last_fullsized":"2024-08-15T17:58:36.287149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:02:13.223698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283315+0000","last_change":"2024-08-15T17:58:36.283315+0000","last_active":"2024-08-15T17:58:36.283315+0000","last_peered":"2024-08-15T17:58:36.283315+0000","last_clean":"2024-08-15T17:58:36.283315+0000","last_became_active":"2024-08-15T17:58:36.282873+0000","last_became_peered":"2024-08-15T17:58:36.282873+0000","last_unstale":"2024-08-15T17:58:36.283315+0000","last_undegraded":"2024-08-15T17:58:36.283315+0000","last_fullsized":"2024-08-15T17:58:36.283315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:31:12.400383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.284677+0000","last_change":"2024-08-15T17:58:36.284677+0000","last_active":"2024-08-15T17:58:36.284677+0000","last_peered":"2024-08-15T17:58:36.284677+0000","last_clean":"2024-08-15T17:58:36.284677+0000","last_became_active":"2024-08-15T17:58:36.281734+0000","last_became_peered":"2024-08-15T17:58:36.281734+0000","last_unstale":"2024-08-15T17:58:36.284677+0000","last_undegraded":"2024-08-15T17:58:36.284677+0000","last_fullsized":"2024-08-15T17:58:36.284677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.281437+0000","last_change":"2024-08-15T17:58:36.281437+0000","last_active":"2024-08-15T17:58:36.281437+0000","last_peered":"2024-08-15T17:58:36.281437+0000","last_clean":"2024-08-15T17:58:36.281437+0000","last_became_active":"2024-08-15T17:58:36.280874+0000","last_became_peered":"2024-08-15T17:58:36.280874+0000","last_unstale":"2024-08-15T17:58:36.281437+0000","last_undegraded":"2024-08-15T17:58:36.281437+0000","last_fullsized":"2024-08-15T17:58:36.281437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:05.015208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285476+0000","last_change":"2024-08-15T17:58:36.285476+0000","last_active":"2024-08-15T17:58:36.285476+0000","last_peered":"2024-08-15T17:58:36.285476+0000","last_clean":"2024-08-15T17:58:36.285476+0000","last_became_active":"2024-08-15T17:58:36.285049+0000","last_became_peered":"2024-08-15T17:58:36.285049+0000","last_unstale":"2024-08-15T17:58:36.285476+0000","last_undegraded":"2024-08-15T17:58:36.285476+0000","last_fullsized":"2024-08-15T17:58:36.285476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:32:58.557897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286646+0000","last_change":"2024-08-15T17:58:36.286646+0000","last_active":"2024-08-15T17:58:36.286646+0000","last_peered":"2024-08-15T17:58:36.286646+0000","last_clean":"2024-08-15T17:58:36.286646+0000","last_became_active":"2024-08-15T17:58:36.286375+0000","last_became_peered":"2024-08-15T17:58:36.286375+0000","last_unstale":"2024-08-15T17:58:36.286646+0000","last_undegraded":"2024-08-15T17:58:36.286646+0000","last_fullsized":"2024-08-15T17:58:36.286646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:33.556488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285304+0000","last_change":"2024-08-15T17:58:36.278354+0000","last_active":"2024-08-15T17:58:36.285304+0000","last_peered":"2024-08-15T17:58:36.285304+0000","last_clean":"2024-08-15T17:58:36.285304+0000","last_became_active":"2024-08-15T17:58:36.278061+0000","last_became_peered":"2024-08-15T17:58:36.278061+0000","last_unstale":"2024-08-15T17:58:36.285304+0000","last_undegraded":"2024-08-15T17:58:36.285304+0000","last_fullsized":"2024-08-15T17:58:36.285304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:45.183805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280217+0000","last_change":"2024-08-15T17:58:35.277170+0000","last_active":"2024-08-15T17:58:36.280217+0000","last_peered":"2024-08-15T17:58:36.280217+0000","last_clean":"2024-08-15T17:58:36.280217+0000","last_became_active":"2024-08-15T17:58:35.276904+0000","last_became_peered":"2024-08-15T17:58:35.276904+0000","last_unstale":"2024-08-15T17:58:36.280217+0000","last_undegraded":"2024-08-15T17:58:36.280217+0000","last_fullsized":"2024-08-15T17:58:36.280217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:11:34.747972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.271912+0000","last_change":"2024-08-15T17:58:36.271912+0000","last_active":"2024-08-15T17:58:36.271912+0000","last_peered":"2024-08-15T17:58:36.271912+0000","last_clean":"2024-08-15T17:58:36.271912+0000","last_became_active":"2024-08-15T17:58:36.271346+0000","last_became_peered":"2024-08-15T17:58:36.271346+0000","last_unstale":"2024-08-15T17:58:36.271912+0000","last_undegraded":"2024-08-15T17:58:36.271912+0000","last_fullsized":"2024-08-15T17:58:36.271912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:54:02.548772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283385+0000","last_change":"2024-08-15T17:58:35.266446+0000","last_active":"2024-08-15T17:58:36.283385+0000","last_peered":"2024-08-15T17:58:36.283385+0000","last_clean":"2024-08-15T17:58:36.283385+0000","last_became_active":"2024-08-15T17:58:35.265867+0000","last_became_peered":"2024-08-15T17:58:35.265867+0000","last_unstale":"2024-08-15T17:58:36.283385+0000","last_undegraded":"2024-08-15T17:58:36.283385+0000","last_fullsized":"2024-08-15T17:58:36.283385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:11:26.241336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.280190+0000","last_change":"2024-08-15T17:58:35.275168+0000","last_active":"2024-08-15T17:58:35.280190+0000","last_peered":"2024-08-15T17:58:35.280190+0000","last_clean":"2024-08-15T17:58:35.280190+0000","last_became_active":"2024-08-15T17:58:35.274863+0000","last_became_peered":"2024-08-15T17:58:35.274863+0000","last_unstale":"2024-08-15T17:58:35.280190+0000","last_undegraded":"2024-08-15T17:58:35.280190+0000","last_fullsized":"2024-08-15T17:58:35.280190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:17:09.213636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290048+0000","last_change":"2024-08-15T17:58:36.290048+0000","last_active":"2024-08-15T17:58:36.290048+0000","last_peered":"2024-08-15T17:58:36.290048+0000","last_clean":"2024-08-15T17:58:36.290048+0000","last_became_active":"2024-08-15T17:58:36.289720+0000","last_became_peered":"2024-08-15T17:58:36.289720+0000","last_unstale":"2024-08-15T17:58:36.290048+0000","last_undegraded":"2024-08-15T17:58:36.290048+0000","last_fullsized":"2024-08-15T17:58:36.290048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:55.518837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.276294+0000","last_change":"2024-08-15T17:58:35.266688+0000","last_active":"2024-08-15T17:58:35.276294+0000","last_peered":"2024-08-15T17:58:35.276294+0000","last_clean":"2024-08-15T17:58:35.276294+0000","last_became_active":"2024-08-15T17:58:35.266306+0000","last_became_peered":"2024-08-15T17:58:35.266306+0000","last_unstale":"2024-08-15T17:58:35.276294+0000","last_undegraded":"2024-08-15T17:58:35.276294+0000","last_fullsized":"2024-08-15T17:58:35.276294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:11:34.747972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.287582+0000","last_change":"2024-08-15T17:58:36.287582+0000","last_active":"2024-08-15T17:58:36.287582+0000","last_peered":"2024-08-15T17:58:36.287582+0000","last_clean":"2024-08-15T17:58:36.287582+0000","last_became_active":"2024-08-15T17:58:36.286980+0000","last_became_peered":"2024-08-15T17:58:36.286980+0000","last_unstale":"2024-08-15T17:58:36.287582+0000","last_undegraded":"2024-08-15T17:58:36.287582+0000","last_fullsized":"2024-08-15T17:58:36.287582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:49:19.919158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.270768+0000","last_change":"2024-08-15T17:58:35.267834+0000","last_active":"2024-08-15T17:58:35.270768+0000","last_peered":"2024-08-15T17:58:35.270768+0000","last_clean":"2024-08-15T17:58:35.270768+0000","last_became_active":"2024-08-15T17:58:35.267249+0000","last_became_peered":"2024-08-15T17:58:35.267249+0000","last_unstale":"2024-08-15T17:58:35.270768+0000","last_undegraded":"2024-08-15T17:58:35.270768+0000","last_fullsized":"2024-08-15T17:58:35.270768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:17.590038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280965+0000","last_change":"2024-08-15T17:58:36.280965+0000","last_active":"2024-08-15T17:58:36.280965+0000","last_peered":"2024-08-15T17:58:36.280965+0000","last_clean":"2024-08-15T17:58:36.280965+0000","last_became_active":"2024-08-15T17:58:36.280684+0000","last_became_peered":"2024-08-15T17:58:36.280684+0000","last_unstale":"2024-08-15T17:58:36.280965+0000","last_undegraded":"2024-08-15T17:58:36.280965+0000","last_fullsized":"2024-08-15T17:58:36.280965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:15:59.759500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.276504+0000","last_change":"2024-08-15T17:58:35.268261+0000","last_active":"2024-08-15T17:58:35.276504+0000","last_peered":"2024-08-15T17:58:35.276504+0000","last_clean":"2024-08-15T17:58:35.276504+0000","last_became_active":"2024-08-15T17:58:35.266109+0000","last_became_peered":"2024-08-15T17:58:35.266109+0000","last_unstale":"2024-08-15T17:58:35.276504+0000","last_undegraded":"2024-08-15T17:58:35.276504+0000","last_fullsized":"2024-08-15T17:58:35.276504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:06:34.826184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283639+0000","last_change":"2024-08-15T17:58:36.283639+0000","last_active":"2024-08-15T17:58:36.283639+0000","last_peered":"2024-08-15T17:58:36.283639+0000","last_clean":"2024-08-15T17:58:36.283639+0000","last_became_active":"2024-08-15T17:58:36.282343+0000","last_became_peered":"2024-08-15T17:58:36.282343+0000","last_unstale":"2024-08-15T17:58:36.283639+0000","last_undegraded":"2024-08-15T17:58:36.283639+0000","last_fullsized":"2024-08-15T17:58:36.283639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:58:52.618737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.300516+0000","last_change":"2024-08-15T17:58:35.276856+0000","last_active":"2024-08-15T17:58:36.300516+0000","last_peered":"2024-08-15T17:58:36.300516+0000","last_clean":"2024-08-15T17:58:36.300516+0000","last_became_active":"2024-08-15T17:58:35.275745+0000","last_became_peered":"2024-08-15T17:58:35.275745+0000","last_unstale":"2024-08-15T17:58:36.300516+0000","last_undegraded":"2024-08-15T17:58:36.300516+0000","last_fullsized":"2024-08-15T17:58:36.300516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:26:33.980860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.279341+0000","last_change":"2024-08-15T17:58:36.272759+0000","last_active":"2024-08-15T17:58:36.279341+0000","last_peered":"2024-08-15T17:58:36.279341+0000","last_clean":"2024-08-15T17:58:36.279341+0000","last_became_active":"2024-08-15T17:58:36.271418+0000","last_became_peered":"2024-08-15T17:58:36.271418+0000","last_unstale":"2024-08-15T17:58:36.279341+0000","last_undegraded":"2024-08-15T17:58:36.279341+0000","last_fullsized":"2024-08-15T17:58:36.279341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:32:58.557897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.300061+0000","last_change":"2024-08-15T17:58:35.271651+0000","last_active":"2024-08-15T17:58:36.300061+0000","last_peered":"2024-08-15T17:58:36.300061+0000","last_clean":"2024-08-15T17:58:36.300061+0000","last_became_active":"2024-08-15T17:58:35.271306+0000","last_became_peered":"2024-08-15T17:58:35.271306+0000","last_unstale":"2024-08-15T17:58:36.300061+0000","last_undegraded":"2024-08-15T17:58:36.300061+0000","last_fullsized":"2024-08-15T17:58:36.300061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:44.174826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.284247+0000","last_change":"2024-08-15T17:58:36.284247+0000","last_active":"2024-08-15T17:58:36.284247+0000","last_peered":"2024-08-15T17:58:36.284247+0000","last_clean":"2024-08-15T17:58:36.284247+0000","last_became_active":"2024-08-15T17:58:36.283981+0000","last_became_peered":"2024-08-15T17:58:36.283981+0000","last_unstale":"2024-08-15T17:58:36.284247+0000","last_undegraded":"2024-08-15T17:58:36.284247+0000","last_fullsized":"2024-08-15T17:58:36.284247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:09:08.360709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.305509+0000","last_change":"2024-08-15T17:58:35.268853+0000","last_active":"2024-08-15T17:58:36.305509+0000","last_peered":"2024-08-15T17:58:36.305509+0000","last_clean":"2024-08-15T17:58:36.305509+0000","last_became_active":"2024-08-15T17:58:35.268541+0000","last_became_peered":"2024-08-15T17:58:35.268541+0000","last_unstale":"2024-08-15T17:58:36.305509+0000","last_undegraded":"2024-08-15T17:58:36.305509+0000","last_fullsized":"2024-08-15T17:58:36.305509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:33:31.329931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.278321+0000","last_change":"2024-08-15T17:58:36.278321+0000","last_active":"2024-08-15T17:58:36.278321+0000","last_peered":"2024-08-15T17:58:36.278321+0000","last_clean":"2024-08-15T17:58:36.278321+0000","last_became_active":"2024-08-15T17:58:36.278014+0000","last_became_peered":"2024-08-15T17:58:36.278014+0000","last_unstale":"2024-08-15T17:58:36.278321+0000","last_undegraded":"2024-08-15T17:58:36.278321+0000","last_fullsized":"2024-08-15T17:58:36.278321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:27:54.650775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.279183+0000","last_change":"2024-08-15T17:58:35.278113+0000","last_active":"2024-08-15T17:58:35.279183+0000","last_peered":"2024-08-15T17:58:35.279183+0000","last_clean":"2024-08-15T17:58:35.279183+0000","last_became_active":"2024-08-15T17:58:35.277811+0000","last_became_peered":"2024-08-15T17:58:35.277811+0000","last_unstale":"2024-08-15T17:58:35.279183+0000","last_undegraded":"2024-08-15T17:58:35.279183+0000","last_fullsized":"2024-08-15T17:58:35.279183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:12:40.711380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283999+0000","last_change":"2024-08-15T17:58:36.283999+0000","last_active":"2024-08-15T17:58:36.283999+0000","last_peered":"2024-08-15T17:58:36.283999+0000","last_clean":"2024-08-15T17:58:36.283999+0000","last_became_active":"2024-08-15T17:58:36.283723+0000","last_became_peered":"2024-08-15T17:58:36.283723+0000","last_unstale":"2024-08-15T17:58:36.283999+0000","last_undegraded":"2024-08-15T17:58:36.283999+0000","last_fullsized":"2024-08-15T17:58:36.283999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:23:04.074155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.270801+0000","last_change":"2024-08-15T17:58:35.268112+0000","last_active":"2024-08-15T17:58:35.270801+0000","last_peered":"2024-08-15T17:58:35.270801+0000","last_clean":"2024-08-15T17:58:35.270801+0000","last_became_active":"2024-08-15T17:58:35.265298+0000","last_became_peered":"2024-08-15T17:58:35.265298+0000","last_unstale":"2024-08-15T17:58:35.270801+0000","last_undegraded":"2024-08-15T17:58:35.270801+0000","last_fullsized":"2024-08-15T17:58:35.270801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:13.047239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286548+0000","last_change":"2024-08-15T17:58:36.286548+0000","last_active":"2024-08-15T17:58:36.286548+0000","last_peered":"2024-08-15T17:58:36.286548+0000","last_clean":"2024-08-15T17:58:36.286548+0000","last_became_active":"2024-08-15T17:58:36.286287+0000","last_became_peered":"2024-08-15T17:58:36.286287+0000","last_unstale":"2024-08-15T17:58:36.286548+0000","last_undegraded":"2024-08-15T17:58:36.286548+0000","last_fullsized":"2024-08-15T17:58:36.286548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:07:03.605391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.304134+0000","last_change":"2024-08-15T17:58:35.271260+0000","last_active":"2024-08-15T17:58:36.304134+0000","last_peered":"2024-08-15T17:58:36.304134+0000","last_clean":"2024-08-15T17:58:36.304134+0000","last_became_active":"2024-08-15T17:58:35.270816+0000","last_became_peered":"2024-08-15T17:58:35.270816+0000","last_unstale":"2024-08-15T17:58:36.304134+0000","last_undegraded":"2024-08-15T17:58:36.304134+0000","last_fullsized":"2024-08-15T17:58:36.304134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:58:46.815165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283169+0000","last_change":"2024-08-15T17:58:36.283169+0000","last_active":"2024-08-15T17:58:36.283169+0000","last_peered":"2024-08-15T17:58:36.283169+0000","last_clean":"2024-08-15T17:58:36.283169+0000","last_became_active":"2024-08-15T17:58:36.282882+0000","last_became_peered":"2024-08-15T17:58:36.282882+0000","last_unstale":"2024-08-15T17:58:36.283169+0000","last_undegraded":"2024-08-15T17:58:36.283169+0000","last_fullsized":"2024-08-15T17:58:36.283169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:24:27.743039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.299612+0000","last_change":"2024-08-15T17:58:35.266260+0000","last_active":"2024-08-15T17:58:36.299612+0000","last_peered":"2024-08-15T17:58:36.299612+0000","last_clean":"2024-08-15T17:58:36.299612+0000","last_became_active":"2024-08-15T17:58:35.265941+0000","last_became_peered":"2024-08-15T17:58:35.265941+0000","last_unstale":"2024-08-15T17:58:36.299612+0000","last_undegraded":"2024-08-15T17:58:36.299612+0000","last_fullsized":"2024-08-15T17:58:36.299612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:48:09.608873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285123+0000","last_change":"2024-08-15T17:58:36.285123+0000","last_active":"2024-08-15T17:58:36.285123+0000","last_peered":"2024-08-15T17:58:36.285123+0000","last_clean":"2024-08-15T17:58:36.285123+0000","last_became_active":"2024-08-15T17:58:36.284782+0000","last_became_peered":"2024-08-15T17:58:36.284782+0000","last_unstale":"2024-08-15T17:58:36.285123+0000","last_undegraded":"2024-08-15T17:58:36.285123+0000","last_fullsized":"2024-08-15T17:58:36.285123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:42:52.458542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.300580+0000","last_change":"2024-08-15T17:58:35.273755+0000","last_active":"2024-08-15T17:58:36.300580+0000","last_peered":"2024-08-15T17:58:36.300580+0000","last_clean":"2024-08-15T17:58:36.300580+0000","last_became_active":"2024-08-15T17:58:35.273318+0000","last_became_peered":"2024-08-15T17:58:35.273318+0000","last_unstale":"2024-08-15T17:58:36.300580+0000","last_undegraded":"2024-08-15T17:58:36.300580+0000","last_fullsized":"2024-08-15T17:58:36.300580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:13.047239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286635+0000","last_change":"2024-08-15T17:58:36.286635+0000","last_active":"2024-08-15T17:58:36.286635+0000","last_peered":"2024-08-15T17:58:36.286635+0000","last_clean":"2024-08-15T17:58:36.286635+0000","last_became_active":"2024-08-15T17:58:36.286062+0000","last_became_peered":"2024-08-15T17:58:36.286062+0000","last_unstale":"2024-08-15T17:58:36.286635+0000","last_undegraded":"2024-08-15T17:58:36.286635+0000","last_fullsized":"2024-08-15T17:58:36.286635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:21:42.887962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.273091+0000","last_change":"2024-08-15T17:58:35.274846+0000","last_active":"2024-08-15T17:58:36.273091+0000","last_peered":"2024-08-15T17:58:36.273091+0000","last_clean":"2024-08-15T17:58:36.273091+0000","last_became_active":"2024-08-15T17:58:35.274543+0000","last_became_peered":"2024-08-15T17:58:35.274543+0000","last_unstale":"2024-08-15T17:58:36.273091+0000","last_undegraded":"2024-08-15T17:58:36.273091+0000","last_fullsized":"2024-08-15T17:58:36.273091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:11:26.241336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285388+0000","last_change":"2024-08-15T17:58:36.285388+0000","last_active":"2024-08-15T17:58:36.285388+0000","last_peered":"2024-08-15T17:58:36.285388+0000","last_clean":"2024-08-15T17:58:36.285388+0000","last_became_active":"2024-08-15T17:58:36.285136+0000","last_became_peered":"2024-08-15T17:58:36.285136+0000","last_unstale":"2024-08-15T17:58:36.285388+0000","last_undegraded":"2024-08-15T17:58:36.285388+0000","last_fullsized":"2024-08-15T17:58:36.285388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:43.158602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.273848+0000","last_change":"2024-08-15T17:58:35.272166+0000","last_active":"2024-08-15T17:58:35.273848+0000","last_peered":"2024-08-15T17:58:35.273848+0000","last_clean":"2024-08-15T17:58:35.273848+0000","last_became_active":"2024-08-15T17:58:35.271841+0000","last_became_peered":"2024-08-15T17:58:35.271841+0000","last_unstale":"2024-08-15T17:58:35.273848+0000","last_undegraded":"2024-08-15T17:58:35.273848+0000","last_fullsized":"2024-08-15T17:58:35.273848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:40:09.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285039+0000","last_change":"2024-08-15T17:58:36.285039+0000","last_active":"2024-08-15T17:58:36.285039+0000","last_peered":"2024-08-15T17:58:36.285039+0000","last_clean":"2024-08-15T17:58:36.285039+0000","last_became_active":"2024-08-15T17:58:36.284458+0000","last_became_peered":"2024-08-15T17:58:36.284458+0000","last_unstale":"2024-08-15T17:58:36.285039+0000","last_undegraded":"2024-08-15T17:58:36.285039+0000","last_fullsized":"2024-08-15T17:58:36.285039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:37:55.417330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.269282+0000","last_change":"2024-08-15T17:58:24.394014+0000","last_active":"2024-08-15T17:58:35.269282+0000","last_peered":"2024-08-15T17:58:35.269282+0000","last_clean":"2024-08-15T17:58:35.269282+0000","last_became_active":"2024-08-15T17:58:24.391735+0000","last_became_peered":"2024-08-15T17:58:24.391735+0000","last_unstale":"2024-08-15T17:58:35.269282+0000","last_undegraded":"2024-08-15T17:58:35.269282+0000","last_fullsized":"2024-08-15T17:58:35.269282+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_clean_scrub_stamp":"2024-08-15T17:58:22.996335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:31.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.276469+0000","last_change":"2024-08-15T17:58:35.269765+0000","last_active":"2024-08-15T17:58:35.276469+0000","last_peered":"2024-08-15T17:58:35.276469+0000","last_clean":"2024-08-15T17:58:35.276469+0000","last_became_active":"2024-08-15T17:58:35.269249+0000","last_became_peered":"2024-08-15T17:58:35.269249+0000","last_unstale":"2024-08-15T17:58:35.276469+0000","last_undegraded":"2024-08-15T17:58:35.276469+0000","last_fullsized":"2024-08-15T17:58:35.276469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:27:34.522329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286437+0000","last_change":"2024-08-15T17:58:36.286437+0000","last_active":"2024-08-15T17:58:36.286437+0000","last_peered":"2024-08-15T17:58:36.286437+0000","last_clean":"2024-08-15T17:58:36.286437+0000","last_became_active":"2024-08-15T17:58:36.285909+0000","last_became_peered":"2024-08-15T17:58:36.285909+0000","last_unstale":"2024-08-15T17:58:36.286437+0000","last_undegraded":"2024-08-15T17:58:36.286437+0000","last_fullsized":"2024-08-15T17:58:36.286437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:47:17.486762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.276920+0000","last_change":"2024-08-15T17:58:35.275547+0000","last_active":"2024-08-15T17:58:35.276920+0000","last_peered":"2024-08-15T17:58:35.276920+0000","last_clean":"2024-08-15T17:58:35.276920+0000","last_became_active":"2024-08-15T17:58:35.275253+0000","last_became_peered":"2024-08-15T17:58:35.275253+0000","last_unstale":"2024-08-15T17:58:35.276920+0000","last_undegraded":"2024-08-15T17:58:35.276920+0000","last_fullsized":"2024-08-15T17:58:35.276920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:44.174826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.277792+0000","last_change":"2024-08-15T17:58:36.277792+0000","last_active":"2024-08-15T17:58:36.277792+0000","last_peered":"2024-08-15T17:58:36.277792+0000","last_clean":"2024-08-15T17:58:36.277792+0000","last_became_active":"2024-08-15T17:58:36.276715+0000","last_became_peered":"2024-08-15T17:58:36.276715+0000","last_unstale":"2024-08-15T17:58:36.277792+0000","last_undegraded":"2024-08-15T17:58:36.277792+0000","last_fullsized":"2024-08-15T17:58:36.277792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:59:27.743235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.281618+0000","last_change":"2024-08-15T17:58:35.280834+0000","last_active":"2024-08-15T17:58:35.281618+0000","last_peered":"2024-08-15T17:58:35.281618+0000","last_clean":"2024-08-15T17:58:35.281618+0000","last_became_active":"2024-08-15T17:58:35.280524+0000","last_became_peered":"2024-08-15T17:58:35.280524+0000","last_unstale":"2024-08-15T17:58:35.281618+0000","last_undegraded":"2024-08-15T17:58:35.281618+0000","last_fullsized":"2024-08-15T17:58:35.281618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:00:50.031787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283772+0000","last_change":"2024-08-15T17:58:36.283772+0000","last_active":"2024-08-15T17:58:36.283772+0000","last_peered":"2024-08-15T17:58:36.283772+0000","last_clean":"2024-08-15T17:58:36.283772+0000","last_became_active":"2024-08-15T17:58:36.283482+0000","last_became_peered":"2024-08-15T17:58:36.283482+0000","last_unstale":"2024-08-15T17:58:36.283772+0000","last_undegraded":"2024-08-15T17:58:36.283772+0000","last_fullsized":"2024-08-15T17:58:36.283772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:33.556488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.276501+0000","last_change":"2024-08-15T17:58:35.270088+0000","last_active":"2024-08-15T17:58:35.276501+0000","last_peered":"2024-08-15T17:58:35.276501+0000","last_clean":"2024-08-15T17:58:35.276501+0000","last_became_active":"2024-08-15T17:58:35.268586+0000","last_became_peered":"2024-08-15T17:58:35.268586+0000","last_unstale":"2024-08-15T17:58:35.276501+0000","last_undegraded":"2024-08-15T17:58:35.276501+0000","last_fullsized":"2024-08-15T17:58:35.276501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:17:09.213636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285313+0000","last_change":"2024-08-15T17:58:36.277552+0000","last_active":"2024-08-15T17:58:36.285313+0000","last_peered":"2024-08-15T17:58:36.285313+0000","last_clean":"2024-08-15T17:58:36.285313+0000","last_became_active":"2024-08-15T17:58:36.277173+0000","last_became_peered":"2024-08-15T17:58:36.277173+0000","last_unstale":"2024-08-15T17:58:36.285313+0000","last_undegraded":"2024-08-15T17:58:36.285313+0000","last_fullsized":"2024-08-15T17:58:36.285313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:18.599017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.272193+0000","last_change":"2024-08-15T17:58:35.270974+0000","last_active":"2024-08-15T17:58:35.272193+0000","last_peered":"2024-08-15T17:58:35.272193+0000","last_clean":"2024-08-15T17:58:35.272193+0000","last_became_active":"2024-08-15T17:58:35.270645+0000","last_became_peered":"2024-08-15T17:58:35.270645+0000","last_unstale":"2024-08-15T17:58:35.272193+0000","last_undegraded":"2024-08-15T17:58:35.272193+0000","last_fullsized":"2024-08-15T17:58:35.272193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:44.174826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.282072+0000","last_change":"2024-08-15T17:58:36.282072+0000","last_active":"2024-08-15T17:58:36.282072+0000","last_peered":"2024-08-15T17:58:36.282072+0000","last_clean":"2024-08-15T17:58:36.282072+0000","last_became_active":"2024-08-15T17:58:36.281850+0000","last_became_peered":"2024-08-15T17:58:36.281850+0000","last_unstale":"2024-08-15T17:58:36.282072+0000","last_undegraded":"2024-08-15T17:58:36.282072+0000","last_fullsized":"2024-08-15T17:58:36.282072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:54:56.996071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.277513+0000","last_change":"2024-08-15T17:58:35.275830+0000","last_active":"2024-08-15T17:58:35.277513+0000","last_peered":"2024-08-15T17:58:35.277513+0000","last_clean":"2024-08-15T17:58:35.277513+0000","last_became_active":"2024-08-15T17:58:35.275532+0000","last_became_peered":"2024-08-15T17:58:35.275532+0000","last_unstale":"2024-08-15T17:58:35.277513+0000","last_undegraded":"2024-08-15T17:58:35.277513+0000","last_fullsized":"2024-08-15T17:58:35.277513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:38.845281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.284264+0000","last_change":"2024-08-15T17:58:36.284264+0000","last_active":"2024-08-15T17:58:36.284264+0000","last_peered":"2024-08-15T17:58:36.284264+0000","last_clean":"2024-08-15T17:58:36.284264+0000","last_became_active":"2024-08-15T17:58:36.283964+0000","last_became_peered":"2024-08-15T17:58:36.283964+0000","last_unstale":"2024-08-15T17:58:36.284264+0000","last_undegraded":"2024-08-15T17:58:36.284264+0000","last_fullsized":"2024-08-15T17:58:36.284264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:07:03.605391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.277468+0000","last_change":"2024-08-15T17:58:35.274546+0000","last_active":"2024-08-15T17:58:35.277468+0000","last_peered":"2024-08-15T17:58:35.277468+0000","last_clean":"2024-08-15T17:58:35.277468+0000","last_became_active":"2024-08-15T17:58:35.274232+0000","last_became_peered":"2024-08-15T17:58:35.274232+0000","last_unstale":"2024-08-15T17:58:35.277468+0000","last_undegraded":"2024-08-15T17:58:35.277468+0000","last_fullsized":"2024-08-15T17:58:35.277468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:17.590038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290821+0000","last_change":"2024-08-15T17:58:36.290821+0000","last_active":"2024-08-15T17:58:36.290821+0000","last_peered":"2024-08-15T17:58:36.290821+0000","last_clean":"2024-08-15T17:58:36.290821+0000","last_became_active":"2024-08-15T17:58:36.290495+0000","last_became_peered":"2024-08-15T17:58:36.290495+0000","last_unstale":"2024-08-15T17:58:36.290821+0000","last_undegraded":"2024-08-15T17:58:36.290821+0000","last_fullsized":"2024-08-15T17:58:36.290821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:43.158602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298703+0000","last_change":"2024-08-15T17:58:35.274034+0000","last_active":"2024-08-15T17:58:36.298703+0000","last_peered":"2024-08-15T17:58:36.298703+0000","last_clean":"2024-08-15T17:58:36.298703+0000","last_became_active":"2024-08-15T17:58:35.273028+0000","last_became_peered":"2024-08-15T17:58:35.273028+0000","last_unstale":"2024-08-15T17:58:36.298703+0000","last_undegraded":"2024-08-15T17:58:36.298703+0000","last_fullsized":"2024-08-15T17:58:36.298703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:26:33.980860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.284750+0000","last_change":"2024-08-15T17:58:36.284750+0000","last_active":"2024-08-15T17:58:36.284750+0000","last_peered":"2024-08-15T17:58:36.284750+0000","last_clean":"2024-08-15T17:58:36.284750+0000","last_became_active":"2024-08-15T17:58:36.284480+0000","last_became_peered":"2024-08-15T17:58:36.284480+0000","last_unstale":"2024-08-15T17:58:36.284750+0000","last_undegraded":"2024-08-15T17:58:36.284750+0000","last_fullsized":"2024-08-15T17:58:36.284750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:55.518837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.275687+0000","last_change":"2024-08-15T17:58:35.272396+0000","last_active":"2024-08-15T17:58:35.275687+0000","last_peered":"2024-08-15T17:58:35.275687+0000","last_clean":"2024-08-15T17:58:35.275687+0000","last_became_active":"2024-08-15T17:58:35.272141+0000","last_became_peered":"2024-08-15T17:58:35.272141+0000","last_unstale":"2024-08-15T17:58:35.275687+0000","last_undegraded":"2024-08-15T17:58:35.275687+0000","last_fullsized":"2024-08-15T17:58:35.275687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:17.590038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286071+0000","last_change":"2024-08-15T17:58:36.278104+0000","last_active":"2024-08-15T17:58:36.286071+0000","last_peered":"2024-08-15T17:58:36.286071+0000","last_clean":"2024-08-15T17:58:36.286071+0000","last_became_active":"2024-08-15T17:58:36.277844+0000","last_became_peered":"2024-08-15T17:58:36.277844+0000","last_unstale":"2024-08-15T17:58:36.286071+0000","last_undegraded":"2024-08-15T17:58:36.286071+0000","last_fullsized":"2024-08-15T17:58:36.286071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:26:34.989839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.278361+0000","last_change":"2024-08-15T17:58:35.277768+0000","last_active":"2024-08-15T17:58:35.278361+0000","last_peered":"2024-08-15T17:58:35.278361+0000","last_clean":"2024-08-15T17:58:35.278361+0000","last_became_active":"2024-08-15T17:58:35.277217+0000","last_became_peered":"2024-08-15T17:58:35.277217+0000","last_unstale":"2024-08-15T17:58:35.278361+0000","last_undegraded":"2024-08-15T17:58:35.278361+0000","last_fullsized":"2024-08-15T17:58:35.278361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:13.047239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289433+0000","last_change":"2024-08-15T17:58:36.289433+0000","last_active":"2024-08-15T17:58:36.289433+0000","last_peered":"2024-08-15T17:58:36.289433+0000","last_clean":"2024-08-15T17:58:36.289433+0000","last_became_active":"2024-08-15T17:58:36.288625+0000","last_became_peered":"2024-08-15T17:58:36.288625+0000","last_unstale":"2024-08-15T17:58:36.289433+0000","last_undegraded":"2024-08-15T17:58:36.289433+0000","last_fullsized":"2024-08-15T17:58:36.289433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:47:03.102185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:58:35.278220+0000","last_change":"2024-08-15T17:58:35.276117+0000","last_active":"2024-08-15T17:58:35.278220+0000","last_peered":"2024-08-15T17:58:35.278220+0000","last_clean":"2024-08-15T17:58:35.278220+0000","last_became_active":"2024-08-15T17:58:35.275707+0000","last_became_peered":"2024-08-15T17:58:35.275707+0000","last_unstale":"2024-08-15T17:58:35.278220+0000","last_undegraded":"2024-08-15T17:58:35.278220+0000","last_fullsized":"2024-08-15T17:58:35.278220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:40:09.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290279+0000","last_change":"2024-08-15T17:58:36.290279+0000","last_active":"2024-08-15T17:58:36.290279+0000","last_peered":"2024-08-15T17:58:36.290279+0000","last_clean":"2024-08-15T17:58:36.290279+0000","last_became_active":"2024-08-15T17:58:36.289946+0000","last_became_peered":"2024-08-15T17:58:36.289946+0000","last_unstale":"2024-08-15T17:58:36.290279+0000","last_undegraded":"2024-08-15T17:58:36.290279+0000","last_fullsized":"2024-08-15T17:58:36.290279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298531+0000","last_change":"2024-08-15T17:58:35.274655+0000","last_active":"2024-08-15T17:58:36.298531+0000","last_peered":"2024-08-15T17:58:36.298531+0000","last_clean":"2024-08-15T17:58:36.298531+0000","last_became_active":"2024-08-15T17:58:35.273860+0000","last_became_peered":"2024-08-15T17:58:35.273860+0000","last_unstale":"2024-08-15T17:58:36.298531+0000","last_undegraded":"2024-08-15T17:58:36.298531+0000","last_fullsized":"2024-08-15T17:58:36.298531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:32:57.548918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285943+0000","last_change":"2024-08-15T17:58:36.285943+0000","last_active":"2024-08-15T17:58:36.285943+0000","last_peered":"2024-08-15T17:58:36.285943+0000","last_clean":"2024-08-15T17:58:36.285943+0000","last_became_active":"2024-08-15T17:58:36.283605+0000","last_became_peered":"2024-08-15T17:58:36.283605+0000","last_unstale":"2024-08-15T17:58:36.285943+0000","last_undegraded":"2024-08-15T17:58:36.285943+0000","last_fullsized":"2024-08-15T17:58:36.285943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:07:28.572500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.271574+0000","last_change":"2024-08-15T17:58:36.271574+0000","last_active":"2024-08-15T17:58:36.271574+0000","last_peered":"2024-08-15T17:58:36.271574+0000","last_clean":"2024-08-15T17:58:36.271574+0000","last_became_active":"2024-08-15T17:58:36.271259+0000","last_became_peered":"2024-08-15T17:58:36.271259+0000","last_unstale":"2024-08-15T17:58:36.271574+0000","last_undegraded":"2024-08-15T17:58:36.271574+0000","last_fullsized":"2024-08-15T17:58:36.271574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298400+0000","last_change":"2024-08-15T17:58:35.277718+0000","last_active":"2024-08-15T17:58:36.298400+0000","last_peered":"2024-08-15T17:58:36.298400+0000","last_clean":"2024-08-15T17:58:36.298400+0000","last_became_active":"2024-08-15T17:58:35.277447+0000","last_became_peered":"2024-08-15T17:58:35.277447+0000","last_unstale":"2024-08-15T17:58:36.298400+0000","last_undegraded":"2024-08-15T17:58:36.298400+0000","last_fullsized":"2024-08-15T17:58:36.298400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:23:08.234362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.281039+0000","last_change":"2024-08-15T17:58:36.281039+0000","last_active":"2024-08-15T17:58:36.281039+0000","last_peered":"2024-08-15T17:58:36.281039+0000","last_clean":"2024-08-15T17:58:36.281039+0000","last_became_active":"2024-08-15T17:58:36.280780+0000","last_became_peered":"2024-08-15T17:58:36.280780+0000","last_unstale":"2024-08-15T17:58:36.281039+0000","last_undegraded":"2024-08-15T17:58:36.281039+0000","last_fullsized":"2024-08-15T17:58:36.281039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:47:03.102185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298491+0000","last_change":"2024-08-15T17:58:35.276674+0000","last_active":"2024-08-15T17:58:36.298491+0000","last_peered":"2024-08-15T17:58:36.298491+0000","last_clean":"2024-08-15T17:58:36.298491+0000","last_became_active":"2024-08-15T17:58:35.276363+0000","last_became_peered":"2024-08-15T17:58:35.276363+0000","last_unstale":"2024-08-15T17:58:36.298491+0000","last_undegraded":"2024-08-15T17:58:36.298491+0000","last_fullsized":"2024-08-15T17:58:36.298491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:20.838352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.282308+0000","last_change":"2024-08-15T17:58:36.282308+0000","last_active":"2024-08-15T17:58:36.282308+0000","last_peered":"2024-08-15T17:58:36.282308+0000","last_clean":"2024-08-15T17:58:36.282308+0000","last_became_active":"2024-08-15T17:58:36.281986+0000","last_became_peered":"2024-08-15T17:58:36.281986+0000","last_unstale":"2024-08-15T17:58:36.282308+0000","last_undegraded":"2024-08-15T17:58:36.282308+0000","last_fullsized":"2024-08-15T17:58:36.282308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:48:28.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280663+0000","last_change":"2024-08-15T17:58:35.274614+0000","last_active":"2024-08-15T17:58:36.280663+0000","last_peered":"2024-08-15T17:58:36.280663+0000","last_clean":"2024-08-15T17:58:36.280663+0000","last_became_active":"2024-08-15T17:58:35.274185+0000","last_became_peered":"2024-08-15T17:58:35.274185+0000","last_unstale":"2024-08-15T17:58:36.280663+0000","last_undegraded":"2024-08-15T17:58:36.280663+0000","last_fullsized":"2024-08-15T17:58:36.280663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:41.642352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290572+0000","last_change":"2024-08-15T17:58:36.290572+0000","last_active":"2024-08-15T17:58:36.290572+0000","last_peered":"2024-08-15T17:58:36.290572+0000","last_clean":"2024-08-15T17:58:36.290572+0000","last_became_active":"2024-08-15T17:58:36.290269+0000","last_became_peered":"2024-08-15T17:58:36.290269+0000","last_unstale":"2024-08-15T17:58:36.290572+0000","last_undegraded":"2024-08-15T17:58:36.290572+0000","last_fullsized":"2024-08-15T17:58:36.290572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:56:49.069407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301913+0000","last_change":"2024-08-15T17:58:35.276392+0000","last_active":"2024-08-15T17:58:36.301913+0000","last_peered":"2024-08-15T17:58:36.301913+0000","last_clean":"2024-08-15T17:58:36.301913+0000","last_became_active":"2024-08-15T17:58:35.276085+0000","last_became_peered":"2024-08-15T17:58:35.276085+0000","last_unstale":"2024-08-15T17:58:36.301913+0000","last_undegraded":"2024-08-15T17:58:36.301913+0000","last_fullsized":"2024-08-15T17:58:36.301913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:10:36.097809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298128,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073712,"statfs":{"total":96636764160,"available":96331481088,"internally_reserved":0,"allocated":794624,"data_stored":657451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5569999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":208896,"data_stored":68065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":208896,"data_stored":66639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9419999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":200704,"data_stored":66603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":208896,"data_stored":68065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301700,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070140,"statfs":{"total":96636764160,"available":96327823360,"internally_reserved":0,"allocated":184320,"data_stored":59750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301724,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070116,"statfs":{"total":96636764160,"available":96327798784,"internally_reserved":0,"allocated":204800,"data_stored":66673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":778240,"data_stored":650118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:58:40.233 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-08-15T17:58:40.233 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-15T17:58:40.233 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-08-15T17:58:40.234 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-08-15T17:58:40.234 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-08-15T17:58:40.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e93ad700 1 Processor -- start 2024-08-15T17:58:40.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e93ad700 1 -- start start 2024-08-15T17:58:40.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e93ad700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f61e40b0050 0x7f61e40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e93ad700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f61e4068ce0 0x7f61e40c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e93ad700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e406a3e0 con 0x7f61e4155fb0 2024-08-15T17:58:40.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e93ad700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e4156a90 con 0x7f61e40b0050 2024-08-15T17:58:40.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e93ad700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e40697d0 con 0x7f61e4068ce0 2024-08-15T17:58:40.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e37fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f61e40b0050 0x7f61e40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e37fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f61e40b0050 0x7f61e40687a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53600/0 (socket says 172.21.15.38:53600) 2024-08-15T17:58:40.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e37fe700 1 -- 172.21.15.38:0/252232155 learned_addr learned my addr 172.21.15.38:0/252232155 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:40.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e8bac700 1 --2- 172.21.15.38:0/252232155 >> v2:172.21.15.104:3300/0 conn(0x7f61e4068ce0 0x7f61e40c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e37fe700 1 -- 172.21.15.38:0/252232155 >> v2:172.21.15.104:3300/0 conn(0x7f61e4068ce0 msgr2=0x7f61e40c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:40.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e37fe700 1 --2- 172.21.15.38:0/252232155 >> v2:172.21.15.104:3300/0 conn(0x7f61e4068ce0 0x7f61e40c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e37fe700 1 -- 172.21.15.38:0/252232155 >> v1:172.21.15.104:6789/0 conn(0x7f61e4155fb0 legacy=0x7f61e4156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e37fe700 1 -- 172.21.15.38:0/252232155 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e4069ab0 con 0x7f61e40b0050 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e37fe700 1 --2- 172.21.15.38:0/252232155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e40b0050 0x7f61e40687a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61e406ea00 tx=0x7f61d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f7eacc8ed2e5e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.342+0000 7f61e2ffd700 1 -- 172.21.15.38:0/252232155 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d800b860 con 0x7f61e40b0050 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e2ffd700 1 -- 172.21.15.38:0/252232155 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d800b9c0 con 0x7f61e40b0050 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e2ffd700 1 -- 172.21.15.38:0/252232155 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d800bd60 con 0x7f61e40b0050 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e93ad700 1 -- 172.21.15.38:0/252232155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e40b0050 msgr2=0x7f61e40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e93ad700 1 --2- 172.21.15.38:0/252232155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e40b0050 0x7f61e40687a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61e406ea00 tx=0x7f61d8009250 comp rx=0 tx=0).stop 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e93ad700 1 -- 172.21.15.38:0/252232155 shutdown_connections 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e93ad700 1 --2- 172.21.15.38:0/252232155 >> v2:172.21.15.104:3300/0 conn(0x7f61e4068ce0 0x7f61e40c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e93ad700 1 --2- 172.21.15.38:0/252232155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e40b0050 0x7f61e40687a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e93ad700 1 -- 172.21.15.38:0/252232155 >> 172.21.15.38:0/252232155 conn(0x7f61e40ccfc0 msgr2=0x7f61e40c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:40.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e93ad700 1 -- 172.21.15.38:0/252232155 shutdown_connections 2024-08-15T17:58:40.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e93ad700 1 -- 172.21.15.38:0/252232155 wait complete. 2024-08-15T17:58:40.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e93ad700 1 Processor -- start 2024-08-15T17:58:40.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.346+0000 7f61e93ad700 1 -- start start 2024-08-15T17:58:40.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e93ad700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61e4068ce0 0x7f61e414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e93ad700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e40b0050 0x7f61e4151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e93ad700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61e4155fb0 0x7f61e41519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e93ad700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e4158580 con 0x7f61e40b0050 2024-08-15T17:58:40.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e93ad700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e41653c0 con 0x7f61e4068ce0 2024-08-15T17:58:40.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e93ad700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40c2a00 con 0x7f61e4155fb0 2024-08-15T17:58:40.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e37fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e40b0050 0x7f61e4151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e37fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e40b0050 0x7f61e4151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53604/0 (socket says 172.21.15.38:53604) 2024-08-15T17:58:40.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e37fe700 1 -- 172.21.15.38:0/4131443665 learned_addr learned my addr 172.21.15.38:0/4131443665 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:40.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e8bac700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61e4155fb0 0x7f61e41519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e37fe700 1 -- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61e4155fb0 msgr2=0x7f61e41519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e37fe700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61e4155fb0 0x7f61e41519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e37fe700 1 -- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61e4068ce0 msgr2=0x7f61e414f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e37fe700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61e4068ce0 0x7f61e414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e37fe700 1 -- 172.21.15.38:0/4131443665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d80094f0 con 0x7f61e40b0050 2024-08-15T17:58:40.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e37fe700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e40b0050 0x7f61e4151490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f61e4069330 tx=0x7f61d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:40.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61cbfff700 1 -- 172.21.15.38:0/4131443665 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8011e60 con 0x7f61e40b0050 2024-08-15T17:58:40.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61cbfff700 1 -- 172.21.15.38:0/4131443665 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d801a630 con 0x7f61e40b0050 2024-08-15T17:58:40.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61cbfff700 1 -- 172.21.15.38:0/4131443665 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d801a9f0 con 0x7f61e40b0050 2024-08-15T17:58:40.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e93ad700 1 -- 172.21.15.38:0/4131443665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e40c2ce0 con 0x7f61e40b0050 2024-08-15T17:58:40.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e93ad700 1 -- 172.21.15.38:0/4131443665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e41655d0 con 0x7f61e40b0050 2024-08-15T17:58:40.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.350+0000 7f61e93ad700 1 -- 172.21.15.38:0/4131443665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e41657f0 con 0x7f61e40b0050 2024-08-15T17:58:40.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.354+0000 7f61cbfff700 1 -- 172.21.15.38:0/4131443665 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f61d802d440 con 0x7f61e40b0050 2024-08-15T17:58:40.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.354+0000 7f61cbfff700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f61d0060860 0x7f61d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.354+0000 7f61cbfff700 1 -- 172.21.15.38:0/4131443665 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f61d807e820 con 0x7f61e40b0050 2024-08-15T17:58:40.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.354+0000 7f61e3fff700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f61d0060860 0x7f61d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.354+0000 7f61e3fff700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f61d0060860 0x7f61d0062d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f61d4001690 tx=0x7f61d4007b40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:40.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.358+0000 7f61cbfff700 1 -- 172.21.15.38:0/4131443665 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f61d804ec70 con 0x7f61e40b0050 2024-08-15T17:58:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.562+0000 7f61e93ad700 1 -- 172.21.15.38:0/4131443665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f61e40638f0 con 0x7f61e40b0050 2024-08-15T17:58:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.566+0000 7f61cbfff700 1 -- 172.21.15.38:0/4131443665 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+154542 (secure 0 0 0) 0x7f61d8052290 con 0x7f61e40b0050 2024-08-15T17:58:40.569 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.566+0000 7f61e93ad700 1 -- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f61d0060860 msgr2=0x7f61d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.566+0000 7f61e93ad700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f61d0060860 0x7f61d0062d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f61d4001690 tx=0x7f61d4007b40 comp rx=0 tx=0).stop 2024-08-15T17:58:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.566+0000 7f61e93ad700 1 -- 172.21.15.38:0/4131443665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e40b0050 msgr2=0x7f61e4151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.570+0000 7f61e93ad700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e40b0050 0x7f61e4151490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f61e4069330 tx=0x7f61d8017000 comp rx=0 tx=0).stop 2024-08-15T17:58:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.570+0000 7f61e93ad700 1 -- 172.21.15.38:0/4131443665 shutdown_connections 2024-08-15T17:58:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.570+0000 7f61e93ad700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f61d0060860 0x7f61d0062d10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.570+0000 7f61e93ad700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61e4155fb0 0x7f61e41519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.570+0000 7f61e93ad700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e40b0050 0x7f61e4151490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.570+0000 7f61e93ad700 1 --2- 172.21.15.38:0/4131443665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61e4068ce0 0x7f61e414f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.570+0000 7f61e93ad700 1 -- 172.21.15.38:0/4131443665 >> 172.21.15.38:0/4131443665 conn(0x7f61e40ccfc0 msgr2=0x7f61e41529c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.570+0000 7f61e93ad700 1 -- 172.21.15.38:0/4131443665 shutdown_connections 2024-08-15T17:58:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.570+0000 7f61e93ad700 1 -- 172.21.15.38:0/4131443665 wait complete. 2024-08-15T17:58:40.583 INFO:tasks.ceph.mds.b.smithi104.stderr:2024-08-15T17:58:40.580+0000 7f6f213e5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T17:58:40.584 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":24161},{"type":"v1","addr":"172.21.15.104:6833","nonce":24161}]},"active_addr":"172.21.15.104:6833/24161","active_change":"2024-08-15T17:58:16.154583+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4105,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":1063497260}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":4090194851}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":4017500614}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":2844237989}]}]} 2024-08-15T17:58:40.587 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-08-15T17:58:40.587 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-08-15T17:58:40.588 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.694+0000 7f4fdf59e700 1 Processor -- start 2024-08-15T17:58:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.694+0000 7f4fdf59e700 1 -- start start 2024-08-15T17:58:40.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fdf59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fe0155760 0x7f4fe00682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fdf59e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe006b6d0 con 0x7f4fe0155390 2024-08-15T17:58:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fdf59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe006b3f0 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fdf59e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0155ba0 con 0x7f4fe0155760 2024-08-15T17:58:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fde59c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4fe0155390 0x7f4fe00c96c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40086/0 (socket says 172.21.15.38:40086) 2024-08-15T17:58:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fde59c700 1 -- 172.21.15.38:0/452877627 learned_addr learned my addr 172.21.15.38:0/452877627 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fded9d700 1 --2- 172.21.15.38:0/452877627 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fe0155760 0x7f4fe00682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fddd9b700 1 -- 172.21.15.38:0/452877627 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542109081 0 0) 0x7f4fe006b6d0 con 0x7f4fe0155390 2024-08-15T17:58:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fddd9b700 1 -- 172.21.15.38:0/452877627 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fcc003570 con 0x7f4fe0155390 2024-08-15T17:58:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fddd9b700 1 -- 172.21.15.38:0/452877627 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151117164 0 0) 0x7f4fe006b3f0 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fded9d700 1 -- 172.21.15.38:0/452877627 >> v1:172.21.15.104:6789/0 conn(0x7f4fe00c9dd0 legacy=0x7f4fe00697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fded9d700 1 -- 172.21.15.38:0/452877627 >> v1:172.21.15.38:6789/0 conn(0x7f4fe0155390 legacy=0x7f4fe00c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fded9d700 1 -- 172.21.15.38:0/452877627 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe00677c0 con 0x7f4fe0155760 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fded9d700 1 --2- 172.21.15.38:0/452877627 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fe0155760 0x7f4fe00682a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009e90 tx=0x7f4fd8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d488758bc39c08f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fddd9b700 1 -- 172.21.15.38:0/452877627 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800e070 con 0x7f4fe0155760 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fddd9b700 1 -- 172.21.15.38:0/452877627 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd800bc30 con 0x7f4fe0155760 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fddd9b700 1 -- 172.21.15.38:0/452877627 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8012430 con 0x7f4fe0155760 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.698+0000 7f4fdf59e700 1 -- 172.21.15.38:0/452877627 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fe0155760 msgr2=0x7f4fe00682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.702+0000 7f4fdf59e700 1 --2- 172.21.15.38:0/452877627 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fe0155760 0x7f4fe00682a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009e90 tx=0x7f4fd8009780 comp rx=0 tx=0).stop 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.702+0000 7f4fdf59e700 1 -- 172.21.15.38:0/452877627 shutdown_connections 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.702+0000 7f4fdf59e700 1 --2- 172.21.15.38:0/452877627 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fe0155760 0x7f4fe00682a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.702+0000 7f4fdf59e700 1 -- 172.21.15.38:0/452877627 >> 172.21.15.38:0/452877627 conn(0x7f4fe00ccfc0 msgr2=0x7f4fe00c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.702+0000 7f4fdf59e700 1 -- 172.21.15.38:0/452877627 shutdown_connections 2024-08-15T17:58:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.702+0000 7f4fdf59e700 1 -- 172.21.15.38:0/452877627 wait complete. 2024-08-15T17:58:40.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.702+0000 7f4fdf59e700 1 Processor -- start 2024-08-15T17:58:40.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.702+0000 7f4fdf59e700 1 -- start start 2024-08-15T17:58:40.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fdf59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fe00c9dd0 0x7f4fe014b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fdf59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fe0155390 0x7f4fe0149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fdf59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe0155760 0x7f4fe013ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fdf59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0141920 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fdf59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0142400 con 0x7f4fe0155760 2024-08-15T17:58:40.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fdf59e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0141b10 con 0x7f4fe0155390 2024-08-15T17:58:40.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fde59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fe00c9dd0 0x7f4fe014b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fde59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fe00c9dd0 0x7f4fe014b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53618/0 (socket says 172.21.15.38:53618) 2024-08-15T17:58:40.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fde59c700 1 -- 172.21.15.38:0/947873280 learned_addr learned my addr 172.21.15.38:0/947873280 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:40.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fded9d700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe0155760 0x7f4fe013ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fd7fff700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fe0155390 0x7f4fe0149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fde59c700 1 -- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fe0155390 msgr2=0x7f4fe0149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fde59c700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fe0155390 0x7f4fe0149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fde59c700 1 -- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe0155760 msgr2=0x7f4fe013ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fde59c700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe0155760 0x7f4fe013ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fde59c700 1 -- 172.21.15.38:0/947873280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd8009a20 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fde59c700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fe00c9dd0 0x7f4fe014b770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000c590 tx=0x7f4fd0007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:40.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fd67fc700 1 -- 172.21.15.38:0/947873280 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0005b30 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fd67fc700 1 -- 172.21.15.38:0/947873280 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd0005c90 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fdf59e700 1 -- 172.21.15.38:0/947873280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe0149980 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fd67fc700 1 -- 172.21.15.38:0/947873280 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0007320 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fdf59e700 1 -- 172.21.15.38:0/947873280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe0142090 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.706+0000 7f4fdf59e700 1 -- 172.21.15.38:0/947873280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe00638f0 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.710+0000 7f4fd67fc700 1 -- 172.21.15.38:0/947873280 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4fd0007690 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.710+0000 7f4fd67fc700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4fc0060860 0x7f4fc0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:40.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.710+0000 7f4fd67fc700 1 -- 172.21.15.38:0/947873280 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4fd007bf10 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.710+0000 7f4fd7fff700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4fc0060860 0x7f4fc0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:40.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.710+0000 7f4fd7fff700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4fc0060860 0x7f4fc0062d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00afed0 tx=0x7f4fc8007ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:40.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.714+0000 7f4fd67fc700 1 -- 172.21.15.38:0/947873280 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4fd004c360 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.878+0000 7f4fdf59e700 1 -- 172.21.15.38:0/947873280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fe006ea50 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.878+0000 7f4fd67fc700 1 -- 172.21.15.38:0/947873280 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f4fd0017090 con 0x7f4fe00c9dd0 2024-08-15T17:58:40.882 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:40.882 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:40.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 -- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4fc0060860 msgr2=0x7f4fc0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4fc0060860 0x7f4fc0062d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00afed0 tx=0x7f4fc8007ea0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 -- 172.21.15.38:0/947873280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fe00c9dd0 msgr2=0x7f4fe014b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:40.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fe00c9dd0 0x7f4fe014b770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000c590 tx=0x7f4fd0007db0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 -- 172.21.15.38:0/947873280 shutdown_connections 2024-08-15T17:58:40.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4fc0060860 0x7f4fc0062d10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe0155760 0x7f4fe013ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fe0155390 0x7f4fe0149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 --2- 172.21.15.38:0/947873280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fe00c9dd0 0x7f4fe014b770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:40.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 -- 172.21.15.38:0/947873280 >> 172.21.15.38:0/947873280 conn(0x7f4fe00ccfc0 msgr2=0x7f4fe006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:40.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 -- 172.21.15.38:0/947873280 shutdown_connections 2024-08-15T17:58:40.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:40.882+0000 7f4fdf59e700 1 -- 172.21.15.38:0/947873280 wait complete. 2024-08-15T17:58:40.901 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-08-15T17:58:40.902 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:41.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.006+0000 7f9c5bfff700 1 Processor -- start 2024-08-15T17:58:41.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.006+0000 7f9c5bfff700 1 -- start start 2024-08-15T17:58:41.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c5c153320 0x7f9c5c1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9c5c0cd1b0 0x7f9c5c15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c15dbd0 con 0x7f9c5c15a190 2024-08-15T17:58:41.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c0ce430 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c15d370 con 0x7f9c5c153320 2024-08-15T17:58:41.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5b7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9c5c0cd1b0 0x7f9c5c15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5affd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c5c153320 0x7f9c5c1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5b7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9c5c0cd1b0 0x7f9c5c15cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:35746/0 (socket says 172.21.15.38:35746) 2024-08-15T17:58:41.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5b7fe700 1 -- 172.21.15.38:0/4133169537 learned_addr learned my addr 172.21.15.38:0/4133169537 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5a7fc700 1 --1- 172.21.15.38:0/4133169537 >> v1:172.21.15.104:6789/0 conn(0x7f9c5c15a190 0x7f9c5c0ccaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:59598/0 (socket says 172.21.15.38:59598) 2024-08-15T17:58:41.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5b7fe700 1 -- 172.21.15.38:0/4133169537 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c5c153320 msgr2=0x7f9c5c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5b7fe700 1 --2- 172.21.15.38:0/4133169537 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c5c153320 0x7f9c5c1536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5b7fe700 1 -- 172.21.15.38:0/4133169537 >> v1:172.21.15.104:6789/0 conn(0x7f9c5c15a190 legacy=0x7f9c5c0ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5b7fe700 1 -- 172.21.15.38:0/4133169537 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c15d610 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5b7fe700 1 --2- 172.21.15.38:0/4133169537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c5c0cd1b0 0x7f9c5c15cc60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9c540099b0 tx=0x7f9c54009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1d707ea5159d62 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c59ffb700 1 -- 172.21.15.38:0/4133169537 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54009ea0 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c59ffb700 1 -- 172.21.15.38:0/4133169537 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5400b860 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c59ffb700 1 -- 172.21.15.38:0/4133169537 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400bc00 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 -- 172.21.15.38:0/4133169537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c5c0cd1b0 msgr2=0x7f9c5c15cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 --2- 172.21.15.38:0/4133169537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c5c0cd1b0 0x7f9c5c15cc60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9c540099b0 tx=0x7f9c54009250 comp rx=0 tx=0).stop 2024-08-15T17:58:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 -- 172.21.15.38:0/4133169537 shutdown_connections 2024-08-15T17:58:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 --2- 172.21.15.38:0/4133169537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c5c0cd1b0 0x7f9c5c15cc60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 --2- 172.21.15.38:0/4133169537 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c5c153320 0x7f9c5c1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 -- 172.21.15.38:0/4133169537 >> 172.21.15.38:0/4133169537 conn(0x7f9c5c0d00a0 msgr2=0x7f9c5c0d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 -- 172.21.15.38:0/4133169537 shutdown_connections 2024-08-15T17:58:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 -- 172.21.15.38:0/4133169537 wait complete. 2024-08-15T17:58:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.010+0000 7f9c5bfff700 1 Processor -- start 2024-08-15T17:58:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5bfff700 1 -- start start 2024-08-15T17:58:41.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c5c0cd1b0 0x7f9c5c14b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c5c153320 0x7f9c5c1502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5bfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c5c15a190 0x7f9c5c16c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5bfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c157930 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5bfff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c157ef0 con 0x7f9c5c153320 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5bfff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c15ddc0 con 0x7f9c5c15a190 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c5c0cd1b0 0x7f9c5c14b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c5c0cd1b0 0x7f9c5c14b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53624/0 (socket says 172.21.15.38:53624) 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5affd700 1 -- 172.21.15.38:0/1388860857 learned_addr learned my addr 172.21.15.38:0/1388860857 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5a7fc700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c5c153320 0x7f9c5c1502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5b7fe700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c5c15a190 0x7f9c5c16c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5affd700 1 -- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c5c15a190 msgr2=0x7f9c5c16c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5affd700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c5c15a190 0x7f9c5c16c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5affd700 1 -- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c5c153320 msgr2=0x7f9c5c1502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5affd700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c5c153320 0x7f9c5c1502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5affd700 1 -- 172.21.15.38:0/1388860857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c540094f0 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5a7fc700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c5c153320 0x7f9c5c1502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:41.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5affd700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c5c0cd1b0 0x7f9c5c14b930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00ec80 tx=0x7f9c4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c52ffd700 1 -- 172.21.15.38:0/1388860857 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c009690 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c52ffd700 1 -- 172.21.15.38:0/1388860857 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c00c850 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5bfff700 1 -- 172.21.15.38:0/1388860857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c16d6e0 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c52ffd700 1 -- 172.21.15.38:0/1388860857 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c00cc10 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.014+0000 7f9c5bfff700 1 -- 172.21.15.38:0/1388860857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c150db0 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.018+0000 7f9c5bfff700 1 -- 172.21.15.38:0/1388860857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c0638f0 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.018+0000 7f9c52ffd700 1 -- 172.21.15.38:0/1388860857 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9c4c013470 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.018+0000 7f9c52ffd700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c3c060890 0x7f9c3c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.018+0000 7f9c52ffd700 1 -- 172.21.15.38:0/1388860857 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c4c07ce30 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.018+0000 7f9c5a7fc700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c3c060890 0x7f9c3c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.018+0000 7f9c5a7fc700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c3c060890 0x7f9c3c062d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c14f890 tx=0x7f9c44001a30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.030+0000 7f9c52ffd700 1 -- 172.21.15.38:0/1388860857 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9c4c050c70 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.194+0000 7f9c5bfff700 1 -- 172.21.15.38:0/1388860857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c5c0c5db0 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.198+0000 7f9c52ffd700 1 -- 172.21.15.38:0/1388860857 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f9c4c050800 con 0x7f9c5c0cd1b0 2024-08-15T17:58:41.201 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:41.201 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 -- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c3c060890 msgr2=0x7f9c3c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c3c060890 0x7f9c3c062d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c14f890 tx=0x7f9c44001a30 comp rx=0 tx=0).stop 2024-08-15T17:58:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 -- 172.21.15.38:0/1388860857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c5c0cd1b0 msgr2=0x7f9c5c14b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c5c0cd1b0 0x7f9c5c14b930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00ec80 tx=0x7f9c4c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 -- 172.21.15.38:0/1388860857 shutdown_connections 2024-08-15T17:58:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c3c060890 0x7f9c3c062d40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c5c15a190 0x7f9c5c16c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c5c153320 0x7f9c5c1502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 --2- 172.21.15.38:0/1388860857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c5c0cd1b0 0x7f9c5c14b930 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 -- 172.21.15.38:0/1388860857 >> 172.21.15.38:0/1388860857 conn(0x7f9c5c0d00a0 msgr2=0x7f9c5c15ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 -- 172.21.15.38:0/1388860857 shutdown_connections 2024-08-15T17:58:41.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.202+0000 7f9c5bfff700 1 -- 172.21.15.38:0/1388860857 wait complete. 2024-08-15T17:58:41.216 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-08-15T17:58:41.217 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-08-15T17:58:41.217 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-08-15T17:58:41.218 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-08-15T17:58:41.218 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-08-15T17:58:41.219 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-08-15T17:58:41.219 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-08-15T17:58:41.220 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-08-15T17:58:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.358+0000 7feeb3360700 1 Processor -- start 2024-08-15T17:58:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.358+0000 7feeb3360700 1 -- start start 2024-08-15T17:58:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb3360700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeac0b02a0 0x7feeac0ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb3360700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7feeac155220 0x7feeac0abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb3360700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac068f20 con 0x7feeac154e50 2024-08-15T17:58:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb3360700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac069aa0 con 0x7feeac155220 2024-08-15T17:58:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb3360700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0ac3e0 con 0x7feeac0b02a0 2024-08-15T17:58:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb235e700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7feeac154e50 0x7feeac0afb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40096/0 (socket says 172.21.15.38:40096) 2024-08-15T17:58:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb235e700 1 -- 172.21.15.38:0/941847618 learned_addr learned my addr 172.21.15.38:0/941847618 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb1b5d700 1 --2- 172.21.15.38:0/941847618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeac0b02a0 0x7feeac0ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb2b5f700 1 --2- 172.21.15.38:0/941847618 >> v2:172.21.15.104:3300/0 conn(0x7feeac155220 0x7feeac0abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb135c700 1 -- 172.21.15.38:0/941847618 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262815920 0 0) 0x7feeac068f20 con 0x7feeac154e50 2024-08-15T17:58:41.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb135c700 1 -- 172.21.15.38:0/941847618 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee98003570 con 0x7feeac154e50 2024-08-15T17:58:41.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb1b5d700 1 -- 172.21.15.38:0/941847618 >> v2:172.21.15.104:3300/0 conn(0x7feeac155220 msgr2=0x7feeac0abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb1b5d700 1 --2- 172.21.15.38:0/941847618 >> v2:172.21.15.104:3300/0 conn(0x7feeac155220 0x7feeac0abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb1b5d700 1 -- 172.21.15.38:0/941847618 >> v1:172.21.15.38:6789/0 conn(0x7feeac154e50 legacy=0x7feeac0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb1b5d700 1 -- 172.21.15.38:0/941847618 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac069170 con 0x7feeac0b02a0 2024-08-15T17:58:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb1b5d700 1 --2- 172.21.15.38:0/941847618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeac0b02a0 0x7feeac0ab960 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0098e0 tx=0x7fee9c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c33aa40770f8ce8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb135c700 1 -- 172.21.15.38:0/941847618 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c011630 con 0x7feeac0b02a0 2024-08-15T17:58:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb135c700 1 -- 172.21.15.38:0/941847618 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c011790 con 0x7feeac0b02a0 2024-08-15T17:58:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.362+0000 7feeb135c700 1 -- 172.21.15.38:0/941847618 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c011b10 con 0x7feeac0b02a0 2024-08-15T17:58:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 -- 172.21.15.38:0/941847618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeac0b02a0 msgr2=0x7feeac0ab960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 --2- 172.21.15.38:0/941847618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeac0b02a0 0x7feeac0ab960 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0098e0 tx=0x7fee9c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 -- 172.21.15.38:0/941847618 shutdown_connections 2024-08-15T17:58:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 --2- 172.21.15.38:0/941847618 >> v2:172.21.15.104:3300/0 conn(0x7feeac155220 0x7feeac0abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 --2- 172.21.15.38:0/941847618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeac0b02a0 0x7feeac0ab960 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 -- 172.21.15.38:0/941847618 >> 172.21.15.38:0/941847618 conn(0x7feeac0678a0 msgr2=0x7feeac067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 -- 172.21.15.38:0/941847618 shutdown_connections 2024-08-15T17:58:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 -- 172.21.15.38:0/941847618 wait complete. 2024-08-15T17:58:41.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd63fff700 1 Processor -- start 2024-08-15T17:58:41.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd63fff700 1 -- start start 2024-08-15T17:58:41.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa95359e700 1 Processor -- start 2024-08-15T17:58:41.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa95359e700 1 -- start start 2024-08-15T17:58:41.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 Processor -- start 2024-08-15T17:58:41.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd63fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd5c0dfd10 0x7fbd5c007310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd63fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd5c0e00e0 0x7fbd5c0e4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd63fff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5c0352b0 con 0x7fbd5c036aa0 2024-08-15T17:58:41.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd63fff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5c007850 con 0x7fbd5c0dfd10 2024-08-15T17:58:41.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd63fff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c006270 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa95359e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa9540b0a90 0x7fa95406df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa95359e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa95406e450 0x7fa95415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa95359e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95406e870 con 0x7fa954157a10 2024-08-15T17:58:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa95359e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9540aed20 con 0x7fa95406e450 2024-08-15T17:58:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa95359e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9540b1020 con 0x7fa9540b0a90 2024-08-15T17:58:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 -- start start 2024-08-15T17:58:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa951d9b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa9540b0a90 0x7fa95406df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd627fc700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd5c0e00e0 0x7fbd5c0e4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd627fc700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd5c0e00e0 0x7fbd5c0e4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:43940/0 (socket says 172.21.15.38:43940) 2024-08-15T17:58:41.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd627fc700 1 -- 172.21.15.38:0/3880467064 learned_addr learned my addr 172.21.15.38:0/3880467064 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa952d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa95406e450 0x7fa95415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa951d9b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa9540b0a90 0x7fa95406df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:43946/0 (socket says 172.21.15.38:43946) 2024-08-15T17:58:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa951d9b700 1 -- 172.21.15.38:0/3268099157 learned_addr learned my addr 172.21.15.38:0/3268099157 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeac0b02a0 0x7feeac1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd627fc700 1 -- 172.21.15.38:0/3880467064 >> v2:172.21.15.38:3300/0 conn(0x7fbd5c0dfd10 msgr2=0x7fbd5c007310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa952d9d700 1 -- 172.21.15.38:0/3268099157 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa9540b0a90 msgr2=0x7fa95406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa952d9d700 1 --2- 172.21.15.38:0/3268099157 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa9540b0a90 0x7fa95406df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa95159a700 1 -- 172.21.15.38:0/3268099157 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485996286 0 0) 0x7fa95406e870 con 0x7fa954157a10 2024-08-15T17:58:41.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa952d9d700 1 -- 172.21.15.38:0/3268099157 >> v1:172.21.15.104:6789/0 conn(0x7fa954157a10 legacy=0x7fa95415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa952d9d700 1 -- 172.21.15.38:0/3268099157 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9540b1300 con 0x7fa95406e450 2024-08-15T17:58:41.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd62ffd700 1 --2- 172.21.15.38:0/3880467064 >> v2:172.21.15.38:3300/0 conn(0x7fbd5c0dfd10 0x7fbd5c007310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd61ffb700 1 -- 172.21.15.38:0/3880467064 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503989452 0 0) 0x7fbd5c0352b0 con 0x7fbd5c036aa0 2024-08-15T17:58:41.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd627fc700 1 --2- 172.21.15.38:0/3880467064 >> v2:172.21.15.38:3300/0 conn(0x7fbd5c0dfd10 0x7fbd5c007310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd627fc700 1 -- 172.21.15.38:0/3880467064 >> v1:172.21.15.104:6789/0 conn(0x7fbd5c036aa0 legacy=0x7fbd5c036f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd627fc700 1 -- 172.21.15.38:0/3880467064 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c006550 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeac154e50 0x7feeac1455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fbd62ffd700 1 --2- 172.21.15.38:0/3880467064 >> v2:172.21.15.38:3300/0 conn(0x7fbd5c0dfd10 0x7fbd5c007310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:58:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb235e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeac0b02a0 0x7feeac1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb235e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeac0b02a0 0x7feeac1538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53640/0 (socket says 172.21.15.38:53640) 2024-08-15T17:58:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb235e700 1 -- 172.21.15.38:0/2528489537 learned_addr learned my addr 172.21.15.38:0/2528489537 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeac155220 0x7feeac145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac13a750 con 0x7feeac0b02a0 2024-08-15T17:58:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac157a20 con 0x7feeac155220 2024-08-15T17:58:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb1b5d700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeac154e50 0x7feeac1455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac157d00 con 0x7feeac154e50 2024-08-15T17:58:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.366+0000 7fa952d9d700 1 --2- 172.21.15.38:0/3268099157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa95406e450 0x7fa95415efc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9480098e0 tx=0x7fa9480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b1ed5301e02d596 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd627fc700 1 --2- 172.21.15.38:0/3880467064 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd5c0e00e0 0x7fbd5c0e4550 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbd64060860 tx=0x7fbd64064ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7a3c2dbe210b7bf server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f2feb7dc700 1 Processor -- start 2024-08-15T17:58:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feeb2b5f700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeac155220 0x7feeac145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0d340700 1 Processor -- start 2024-08-15T17:58:41.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0d340700 1 -- start start 2024-08-15T17:58:41.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f2feb7dc700 1 -- start start 2024-08-15T17:58:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd61ffb700 1 -- 172.21.15.38:0/3880467064 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6402b0f0 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd61ffb700 1 -- 172.21.15.38:0/3880467064 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6404c190 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd63fff700 1 -- 172.21.15.38:0/3880467064 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd5c0e00e0 msgr2=0x7fbd5c0e4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd63fff700 1 --2- 172.21.15.38:0/3880467064 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd5c0e00e0 0x7fbd5c0e4550 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbd64060860 tx=0x7fbd64064ee0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feeb1b5d700 1 -- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeac155220 msgr2=0x7feeac145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feeb1b5d700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeac155220 0x7feeac145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feeb1b5d700 1 -- 172.21.15.38:0/2528489537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeac0b02a0 msgr2=0x7feeac1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feeb1b5d700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeac0b02a0 0x7feeac1538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feeb1b5d700 1 -- 172.21.15.38:0/2528489537 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c009420 con 0x7feeac154e50 2024-08-15T17:58:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fa95159a700 1 -- 172.21.15.38:0/3268099157 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94800b640 con 0x7fa95406e450 2024-08-15T17:58:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd63fff700 1 -- 172.21.15.38:0/3880467064 shutdown_connections 2024-08-15T17:58:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd63fff700 1 --2- 172.21.15.38:0/3880467064 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd5c0e00e0 0x7fbd5c0e4550 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd63fff700 1 --2- 172.21.15.38:0/3880467064 >> v2:172.21.15.38:3300/0 conn(0x7fbd5c0dfd10 0x7fbd5c007310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd63fff700 1 -- 172.21.15.38:0/3880467064 >> 172.21.15.38:0/3880467064 conn(0x7fbd5c00a030 msgr2=0x7fbd5c00a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd63fff700 1 -- 172.21.15.38:0/3880467064 shutdown_connections 2024-08-15T17:58:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd63fff700 1 -- 172.21.15.38:0/3880467064 wait complete. 2024-08-15T17:58:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feeb1b5d700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeac154e50 0x7feeac1455a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fee9c016000 tx=0x7fee9c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fa95159a700 1 -- 172.21.15.38:0/3268099157 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94800b7a0 con 0x7fa95406e450 2024-08-15T17:58:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fa95159a700 1 -- 172.21.15.38:0/3268099157 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94800bb40 con 0x7fa95406e450 2024-08-15T17:58:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fa95359e700 1 -- 172.21.15.38:0/3268099157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa95406e450 msgr2=0x7fa95415efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fa95359e700 1 --2- 172.21.15.38:0/3268099157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa95406e450 0x7fa95415efc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9480098e0 tx=0x7fa9480091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fa95359e700 1 -- 172.21.15.38:0/3268099157 shutdown_connections 2024-08-15T17:58:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f2feb7dc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2fe41591c0 0x7f2fe40686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f2feb7dc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2fe4068be0 0x7f2fe40c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f2feb7dc700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe40adfc0 con 0x7f2fe4159590 2024-08-15T17:58:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f2feb7dc700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe4069000 con 0x7f2fe41591c0 2024-08-15T17:58:41.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f2feb7dc700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe40c9350 con 0x7f2fe4068be0 2024-08-15T17:58:41.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fa95359e700 1 --2- 172.21.15.38:0/3268099157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa95406e450 0x7fa95415efc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fa95359e700 1 --2- 172.21.15.38:0/3268099157 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa9540b0a90 0x7fa95406df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fa95359e700 1 -- 172.21.15.38:0/3268099157 >> 172.21.15.38:0/3268099157 conn(0x7fa9540678d0 msgr2=0x7fa954067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fa95359e700 1 -- 172.21.15.38:0/3268099157 shutdown_connections 2024-08-15T17:58:41.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fa95359e700 1 -- 172.21.15.38:0/3268099157 wait complete. 2024-08-15T17:58:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0d340700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1c080b14a0 0x7f1c08157150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0d340700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1c080ad750 0x7f1c080adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0d340700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c081580e0 con 0x7f1c08156d80 2024-08-15T17:58:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0d340700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c081577b0 con 0x7f1c080b14a0 2024-08-15T17:58:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0d340700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08157a90 con 0x7f1c080ad750 2024-08-15T17:58:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0cb3f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1c080ad750 0x7f1c080adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0cb3f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1c080ad750 0x7f1c080adbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:43968/0 (socket says 172.21.15.38:43968) 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c077fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1c080b14a0 0x7f1c08157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0cb3f700 1 -- 172.21.15.38:0/963340843 learned_addr learned my addr 172.21.15.38:0/963340843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c06ffd700 1 -- 172.21.15.38:0/963340843 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3872998123 0 0) 0x7f1c081580e0 con 0x7f1c08156d80 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0cb3f700 1 -- 172.21.15.38:0/963340843 >> v2:172.21.15.104:3300/0 conn(0x7f1c080b14a0 msgr2=0x7f1c08157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0cb3f700 1 --2- 172.21.15.38:0/963340843 >> v2:172.21.15.104:3300/0 conn(0x7f1c080b14a0 0x7f1c08157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0cb3f700 1 -- 172.21.15.38:0/963340843 >> v1:172.21.15.38:6789/0 conn(0x7f1c08156d80 legacy=0x7f1c080b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0cb3f700 1 -- 172.21.15.38:0/963340843 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c08157d70 con 0x7f1c080ad750 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c0cb3f700 1 --2- 172.21.15.38:0/963340843 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1c080ad750 0x7f1c080adbf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0099b0 tx=0x7f1bfc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c870be3fa1fcce1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c06ffd700 1 -- 172.21.15.38:0/963340843 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc009ea0 con 0x7f1c080ad750 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c06ffd700 1 -- 172.21.15.38:0/963340843 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc00b860 con 0x7f1c080ad750 2024-08-15T17:58:41.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7f1c06ffd700 1 -- 172.21.15.38:0/963340843 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00bc00 con 0x7f1c080ad750 2024-08-15T17:58:41.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7fbd63fff700 1 Processor -- start 2024-08-15T17:58:41.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feea27fc700 1 -- 172.21.15.38:0/2528489537 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c011900 con 0x7feeac154e50 2024-08-15T17:58:41.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95359e700 1 Processor -- start 2024-08-15T17:58:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f1c0d340700 1 -- 172.21.15.38:0/963340843 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1c080ad750 msgr2=0x7f1c080adbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f1c0d340700 1 --2- 172.21.15.38:0/963340843 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1c080ad750 0x7f1c080adbf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0099b0 tx=0x7f1bfc009250 comp rx=0 tx=0).stop 2024-08-15T17:58:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2fe9578700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2fe41591c0 0x7f2fe40686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2fe9578700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2fe41591c0 0x7f2fe40686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53650/0 (socket says 172.21.15.38:53650) 2024-08-15T17:58:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2fe9578700 1 -- 172.21.15.38:0/60060706 learned_addr learned my addr 172.21.15.38:0/60060706 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95359e700 1 -- start start 2024-08-15T17:58:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2fe9578700 1 -- 172.21.15.38:0/60060706 >> v2:172.21.15.104:3300/0 conn(0x7f2fe4068be0 msgr2=0x7f2fe40c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2fe9578700 1 --2- 172.21.15.38:0/60060706 >> v2:172.21.15.104:3300/0 conn(0x7f2fe4068be0 0x7f2fe40c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2fe9578700 1 -- 172.21.15.38:0/60060706 >> v1:172.21.15.38:6789/0 conn(0x7f2fe4159590 legacy=0x7f2fe40c8700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T17:58:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feea27fc700 1 -- 172.21.15.38:0/2528489537 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c011a60 con 0x7feeac154e50 2024-08-15T17:58:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feea27fc700 1 -- 172.21.15.38:0/2528489537 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c01a680 con 0x7feeac154e50 2024-08-15T17:58:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac157fe0 con 0x7feeac154e50 2024-08-15T17:58:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.370+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac1461d0 con 0x7feeac154e50 2024-08-15T17:58:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2fe9578700 1 -- 172.21.15.38:0/60060706 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe40ad3b0 con 0x7f2fe41591c0 2024-08-15T17:58:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f1c0d340700 1 -- 172.21.15.38:0/963340843 shutdown_connections 2024-08-15T17:58:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f1c0d340700 1 --2- 172.21.15.38:0/963340843 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1c080ad750 0x7f1c080adbf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f1c0d340700 1 --2- 172.21.15.38:0/963340843 >> v2:172.21.15.104:3300/0 conn(0x7f1c080b14a0 0x7f1c08157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f1c0d340700 1 -- 172.21.15.38:0/963340843 >> 172.21.15.38:0/963340843 conn(0x7f1c080678d0 msgr2=0x7f1c08067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2fe9578700 1 --2- 172.21.15.38:0/60060706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2fe41591c0 0x7f2fe40686a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2fe406d2b0 tx=0x7f2fe0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d17fc448f074efa server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f1c0d340700 1 -- 172.21.15.38:0/963340843 shutdown_connections 2024-08-15T17:58:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95359e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa95406e450 0x7fa95415b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95359e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9540b0a90 0x7fa9541461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95359e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa954157a10 0x7fa95416d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95359e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95415a030 con 0x7fa954157a10 2024-08-15T17:58:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd63fff700 1 -- start start 2024-08-15T17:58:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd63fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd5c036aa0 0x7fbd5c0f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd63fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd5c0dfd10 0x7fbd5c04bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd63fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd5c0e00e0 0x7fbd5c0f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd63fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c0f28c0 con 0x7fbd5c0dfd10 2024-08-15T17:58:41.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd63fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c04cda0 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd63fff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c04c500 con 0x7fbd5c036aa0 2024-08-15T17:58:41.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2fdbfff700 1 -- 172.21.15.38:0/60060706 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe000b820 con 0x7f2fe41591c0 2024-08-15T17:58:41.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2fdbfff700 1 -- 172.21.15.38:0/60060706 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe000b980 con 0x7f2fe41591c0 2024-08-15T17:58:41.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95359e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95415a310 con 0x7fa9540b0a90 2024-08-15T17:58:41.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd627fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd5c0dfd10 0x7fbd5c04bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd627fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd5c0dfd10 0x7fbd5c04bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53674/0 (socket says 172.21.15.38:53674) 2024-08-15T17:58:41.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd627fc700 1 -- 172.21.15.38:0/2348640074 learned_addr learned my addr 172.21.15.38:0/2348640074 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd637fe700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd5c0e00e0 0x7fbd5c0f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95359e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95415a5f0 con 0x7fa95406e450 2024-08-15T17:58:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f1c0d340700 1 -- 172.21.15.38:0/963340843 wait complete. 2024-08-15T17:58:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa952d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa954157a10 0x7fa95416d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7feea27fc700 1 -- 172.21.15.38:0/2528489537 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fee9c01aa70 con 0x7feeac154e50 2024-08-15T17:58:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7feea27fc700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fee900609b0 0x7fee90062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd62ffd700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd5c036aa0 0x7fbd5c0f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa952d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa954157a10 0x7fa95416d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53658/0 (socket says 172.21.15.38:53658) 2024-08-15T17:58:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7feea27fc700 1 -- 172.21.15.38:0/2528489537 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fee9c07e5c0 con 0x7feeac154e50 2024-08-15T17:58:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd637fe700 1 -- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd5c036aa0 msgr2=0x7fbd5c0f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd637fe700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd5c036aa0 0x7fbd5c0f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd637fe700 1 -- 172.21.15.38:0/2348640074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd5c0dfd10 msgr2=0x7fbd5c04bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd637fe700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd5c0dfd10 0x7fbd5c04bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd637fe700 1 -- 172.21.15.38:0/2348640074 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c04c7e0 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7feeb235e700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fee900609b0 0x7fee90062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa952d9d700 1 -- 172.21.15.38:0/431782136 learned_addr learned my addr 172.21.15.38:0/431782136 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95259c700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa95406e450 0x7fa95415b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa951d9b700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9540b0a90 0x7fa9541461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95259c700 1 -- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9540b0a90 msgr2=0x7fa9541461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95259c700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9540b0a90 0x7fa9541461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95259c700 1 -- 172.21.15.38:0/431782136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa954157a10 msgr2=0x7fa95416d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95259c700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa954157a10 0x7fa95416d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95259c700 1 -- 172.21.15.38:0/431782136 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948009420 con 0x7fa95406e450 2024-08-15T17:58:41.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa952d9d700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa954157a10 0x7fa95416d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:41.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2feb7dc700 1 -- 172.21.15.38:0/60060706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2fe41591c0 msgr2=0x7f2fe40686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7f2feb7dc700 1 --2- 172.21.15.38:0/60060706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2fe41591c0 0x7f2fe40686a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2fe406d2b0 tx=0x7f2fe0009250 comp rx=0 tx=0).stop 2024-08-15T17:58:41.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95259c700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa95406e450 0x7fa95415b0d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c490 tx=0x7fa944007ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7feeb3360700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] conn(0x7feeac153e00 0x7feeac155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 --> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feeac0643a0 con 0x7feeac153e00 2024-08-15T17:58:41.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa9427fc700 1 -- 172.21.15.38:0/431782136 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94400ce40 con 0x7fa95406e450 2024-08-15T17:58:41.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95359e700 1 -- 172.21.15.38:0/431782136 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95415a8d0 con 0x7fa95406e450 2024-08-15T17:58:41.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7feeb2b5f700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] conn(0x7feeac153e00 0x7feeac155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fbd637fe700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd5c0e00e0 0x7fbd5c0f5070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c490 tx=0x7fbd58007e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa95359e700 1 -- 172.21.15.38:0/431782136 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa954146720 con 0x7fa95406e450 2024-08-15T17:58:41.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa9427fc700 1 -- 172.21.15.38:0/431782136 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa944011590 con 0x7fa95406e450 2024-08-15T17:58:41.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.374+0000 7fa9427fc700 1 -- 172.21.15.38:0/431782136 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944011950 con 0x7fa95406e450 2024-08-15T17:58:41.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd4affd700 1 -- 172.21.15.38:0/2348640074 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd58012420 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd4affd700 1 -- 172.21.15.38:0/2348640074 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd580125c0 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd4affd700 1 -- 172.21.15.38:0/2348640074 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd58012980 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7feeb235e700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fee900609b0 0x7fee90062e60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feea400f3c0 tx=0x7feea4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f2feb7dc700 1 -- 172.21.15.38:0/60060706 shutdown_connections 2024-08-15T17:58:41.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f2feb7dc700 1 --2- 172.21.15.38:0/60060706 >> v2:172.21.15.104:3300/0 conn(0x7f2fe4068be0 0x7f2fe40c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f2feb7dc700 1 --2- 172.21.15.38:0/60060706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2fe41591c0 0x7f2fe40686a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f2feb7dc700 1 -- 172.21.15.38:0/60060706 >> 172.21.15.38:0/60060706 conn(0x7f2fe406d800 msgr2=0x7f2fe406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd63fff700 1 -- 172.21.15.38:0/2348640074 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5c04cac0 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd63fff700 1 -- 172.21.15.38:0/2348640074 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c04c210 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c0d340700 1 Processor -- start 2024-08-15T17:58:41.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c0d340700 1 -- start start 2024-08-15T17:58:41.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7feeb2b5f700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] conn(0x7feeac153e00 0x7feeac155720 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fa95359e700 1 -- 172.21.15.38:0/431782136 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa938000fd0 con 0x7fa95406e450 2024-08-15T17:58:41.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f2feb7dc700 1 -- 172.21.15.38:0/60060706 shutdown_connections 2024-08-15T17:58:41.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f2feb7dc700 1 -- 172.21.15.38:0/60060706 wait complete. 2024-08-15T17:58:41.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c0d340700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c080ad750 0x7f1c080afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c0d340700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1c080b14a0 0x7f1c08146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c0d340700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c08156d80 0x7f1c080cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c0d340700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080b0bb0 con 0x7f1c08156d80 2024-08-15T17:58:41.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c0d340700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08155840 con 0x7f1c080ad750 2024-08-15T17:58:41.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c0d340700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0816ea40 con 0x7f1c080b14a0 2024-08-15T17:58:41.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c07fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c080ad750 0x7f1c080afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c07fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c080ad750 0x7f1c080afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:35840/0 (socket says 172.21.15.38:35840) 2024-08-15T17:58:41.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c07fff700 1 -- 172.21.15.38:0/3847934128 learned_addr learned my addr 172.21.15.38:0/3847934128 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c077fe700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1c080b14a0 0x7f1c08146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fa9427fc700 1 -- 172.21.15.38:0/431782136 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa944011cc0 con 0x7fa95406e450 2024-08-15T17:58:41.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7feea27fc700 1 -- 172.21.15.38:0/2528489537 <== osd.5 v2:172.21.15.104:6800/24498 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7feeac0643a0 con 0x7feeac153e00 2024-08-15T17:58:41.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f2feb7dc700 1 Processor -- start 2024-08-15T17:58:41.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c07fff700 1 -- 172.21.15.38:0/3847934128 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1c080b14a0 msgr2=0x7f1c08146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c07fff700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1c080b14a0 0x7f1c08146600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c07fff700 1 -- 172.21.15.38:0/3847934128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c08156d80 msgr2=0x7f1c080cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c07fff700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c08156d80 0x7f1c080cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f1c07fff700 1 -- 172.21.15.38:0/3847934128 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc0094f0 con 0x7f1c080ad750 2024-08-15T17:58:41.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd4affd700 1 -- 172.21.15.38:0/2348640074 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbd5801b3b0 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd63fff700 1 -- 172.21.15.38:0/2348640074 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd44000fd0 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd4affd700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbd50060860 0x7fbd50062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd4affd700 1 -- 172.21.15.38:0/2348640074 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbd5807beb0 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f2d6cee5700 1 Processor -- start 2024-08-15T17:58:41.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f2d6cee5700 1 -- start start 2024-08-15T17:58:41.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd4affd700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] conn(0x7fbd50068a70 0x7fbd5006ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd4affd700 1 -- 172.21.15.38:0/2348640074 --> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd5006b530 con 0x7fbd50068a70 2024-08-15T17:58:41.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd4affd700 1 -- 172.21.15.38:0/2348640074 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd5807e4f0 con 0x7fbd5c0e00e0 2024-08-15T17:58:41.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7f2feb7dc700 1 -- start start 2024-08-15T17:58:41.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd62ffd700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbd50060860 0x7fbd50062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fbd627fc700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] conn(0x7fbd50068a70 0x7fbd5006ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7fbd627fc700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] conn(0x7fbd50068a70 0x7fbd5006ae80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4820e68700 1 Processor -- start 2024-08-15T17:58:41.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7fbd62ffd700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbd50060860 0x7fbd50062d10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400adc0 tx=0x7fbd54006f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4820e68700 1 -- start start 2024-08-15T17:58:41.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d6cee5700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d68157a10 0x7f2d6815e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d6cee5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2d680b0a90 0x7f2d6806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2feb7dc700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fe4068be0 0x7f2fe40a8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2feb7dc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fe41591c0 0x7f2fe40a8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2feb7dc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2fe4159590 0x7f2fe40c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2feb7dc700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40a9060 con 0x7f2fe4159590 2024-08-15T17:58:41.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2feb7dc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40a9340 con 0x7f2fe41591c0 2024-08-15T17:58:41.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2feb7dc700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40a9620 con 0x7f2fe4068be0 2024-08-15T17:58:41.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d6cee5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6806e870 con 0x7f2d6806e450 2024-08-15T17:58:41.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d6cee5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6815ea40 con 0x7f2d680b0a90 2024-08-15T17:58:41.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d6cee5700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680aed20 con 0x7f2d68157a10 2024-08-15T17:58:41.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d677fe700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d68157a10 0x7f2d6815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fe9578700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fe4068be0 0x7f2fe40a8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fe9578700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fe4068be0 0x7f2fe40a8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:44020/0 (socket says 172.21.15.38:44020) 2024-08-15T17:58:41.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fe9578700 1 -- 172.21.15.38:0/709452479 learned_addr learned my addr 172.21.15.38:0/709452479 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d66ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2d680b0a90 0x7f2d6806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d677fe700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d68157a10 0x7f2d6815e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:44006/0 (socket says 172.21.15.38:44006) 2024-08-15T17:58:41.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d677fe700 1 -- 172.21.15.38:0/776432635 learned_addr learned my addr 172.21.15.38:0/776432635 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fe8d77700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fe41591c0 0x7f2fe40a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f575183c700 1 Processor -- start 2024-08-15T17:58:41.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7fbd4affd700 1 -- 172.21.15.38:0/2348640074 <== osd.1 v2:172.21.15.38:6816/24122 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbd5006b530 con 0x7fbd50068a70 2024-08-15T17:58:41.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f575183c700 1 -- start start 2024-08-15T17:58:41.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fe9578700 1 -- 172.21.15.38:0/709452479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fe41591c0 msgr2=0x7f2fe40a8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fe9578700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fe41591c0 0x7f2fe40a8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fe9578700 1 -- 172.21.15.38:0/709452479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2fe4159590 msgr2=0x7f2fe40c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fe9578700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2fe4159590 0x7f2fe40c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fe9578700 1 -- 172.21.15.38:0/709452479 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00094f0 con 0x7f2fe4068be0 2024-08-15T17:58:41.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4820e68700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f481c1593b0 0x7f481c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4820e68700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f481c15a8b0 0x7f481c0aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4820e68700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f481c0aa980 0x7f481c0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4820e68700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481c0acae0 con 0x7f481c15a8b0 2024-08-15T17:58:41.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4820e68700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481c0aade0 con 0x7f481c0aa980 2024-08-15T17:58:41.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4820e68700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c0ac650 con 0x7f481c1593b0 2024-08-15T17:58:41.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f481a59c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f481c1593b0 0x7f481c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f481a59c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f481c1593b0 0x7f481c15a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:44032/0 (socket says 172.21.15.38:44032) 2024-08-15T17:58:41.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f481a59c700 1 -- 172.21.15.38:0/524128191 learned_addr learned my addr 172.21.15.38:0/524128191 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4819d9b700 1 --2- 172.21.15.38:0/524128191 >> v2:172.21.15.38:3300/0 conn(0x7f481c15a8b0 0x7f481c0aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fe9578700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fe4068be0 0x7f2fe40a8290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009290 tx=0x7f2fe0016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d66ffd700 1 -- 172.21.15.38:0/776432635 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d68157a10 msgr2=0x7f2d6815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d66ffd700 1 --2- 172.21.15.38:0/776432635 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d68157a10 0x7f2d6815e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d66ffd700 1 -- 172.21.15.38:0/776432635 >> v1:172.21.15.38:6789/0 conn(0x7f2d6806e450 legacy=0x7f2d68161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d66ffd700 1 -- 172.21.15.38:0/776432635 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d680b1020 con 0x7f2d680b0a90 2024-08-15T17:58:41.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d677fe700 1 --2- 172.21.15.38:0/776432635 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d68157a10 0x7f2d6815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:41.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4819d9b700 1 -- 172.21.15.38:0/524128191 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f481c1593b0 msgr2=0x7f481c15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4819d9b700 1 --2- 172.21.15.38:0/524128191 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f481c1593b0 0x7f481c15a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4819d9b700 1 -- 172.21.15.38:0/524128191 >> v2:172.21.15.104:3300/0 conn(0x7f481c0aa980 msgr2=0x7f481c0b0d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T17:58:41.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4819d9b700 1 --2- 172.21.15.38:0/524128191 >> v2:172.21.15.104:3300/0 conn(0x7f481c0aa980 0x7f481c0b0d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f4819d9b700 1 -- 172.21.15.38:0/524128191 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c159780 con 0x7f481c15a8b0 2024-08-15T17:58:41.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f481a59c700 1 --2- 172.21.15.38:0/524128191 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f481c1593b0 0x7f481c15a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:41.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2d66ffd700 1 --2- 172.21.15.38:0/776432635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d680b0a90 0x7f2d6806df10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009e60 tx=0x7f2d60009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45de65a1804c5373 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fd97fa700 1 -- 172.21.15.38:0/709452479 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe001a650 con 0x7f2fe4068be0 2024-08-15T17:58:41.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fd97fa700 1 -- 172.21.15.38:0/709452479 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe001f070 con 0x7f2fe4068be0 2024-08-15T17:58:41.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2fd97fa700 1 -- 172.21.15.38:0/709452479 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe001aa20 con 0x7f2fe4068be0 2024-08-15T17:58:41.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2feb7dc700 1 -- 172.21.15.38:0/709452479 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe40a9900 con 0x7f2fe4068be0 2024-08-15T17:58:41.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f2feb7dc700 1 -- 172.21.15.38:0/709452479 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe40a8d50 con 0x7f2fe4068be0 2024-08-15T17:58:41.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f2d667fc700 1 -- 172.21.15.38:0/776432635 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000e070 con 0x7f2d680b0a90 2024-08-15T17:58:41.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f2d667fc700 1 -- 172.21.15.38:0/776432635 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6000bb20 con 0x7f2d680b0a90 2024-08-15T17:58:41.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f2d667fc700 1 -- 172.21.15.38:0/776432635 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60012430 con 0x7f2d680b0a90 2024-08-15T17:58:41.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f4819d9b700 1 --2- 172.21.15.38:0/524128191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f481c15a8b0 0x7f481c0aa440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f481c04c830 tx=0x7f480c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104abda5f4a71659 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f481959a700 1 -- 172.21.15.38:0/524128191 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c011630 con 0x7f481c15a8b0 2024-08-15T17:58:41.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f481959a700 1 -- 172.21.15.38:0/524128191 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c011790 con 0x7f481c15a8b0 2024-08-15T17:58:41.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f481959a700 1 -- 172.21.15.38:0/524128191 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c011b30 con 0x7f481c15a8b0 2024-08-15T17:58:41.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f2d6cee5700 1 -- 172.21.15.38:0/776432635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d680b0a90 msgr2=0x7f2d6806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f2d6cee5700 1 --2- 172.21.15.38:0/776432635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d680b0a90 0x7f2d6806df10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009e60 tx=0x7f2d60009780 comp rx=0 tx=0).stop 2024-08-15T17:58:41.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f4820e68700 1 -- 172.21.15.38:0/524128191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f481c15a8b0 msgr2=0x7f481c0aa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f4820e68700 1 --2- 172.21.15.38:0/524128191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f481c15a8b0 0x7f481c0aa440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f481c04c830 tx=0x7f480c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f2fd97fa700 1 -- 172.21.15.38:0/709452479 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2fe0026480 con 0x7f2fe4068be0 2024-08-15T17:58:41.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f2fd97fa700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2fc8060860 0x7f2fc8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f2fd97fa700 1 -- 172.21.15.38:0/709452479 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fe0021070 con 0x7f2fe4068be0 2024-08-15T17:58:41.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f2fe8d77700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2fc8060860 0x7f2fc8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f2fe8d77700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2fc8060860 0x7f2fc8062d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009ab0 tx=0x7f2fd4009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fa9427fc700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa934060890 0x7fa934062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fa9427fc700 1 -- 172.21.15.38:0/431782136 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa94407c690 con 0x7fa95406e450 2024-08-15T17:58:41.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fa9427fc700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] conn(0x7fa934068aa0 0x7fa93406aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fa9427fc700 1 -- 172.21.15.38:0/431782136 --> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa93406b560 con 0x7fa934068aa0 2024-08-15T17:58:41.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fa9427fc700 1 -- 172.21.15.38:0/431782136 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa94407ebf0 con 0x7fa95406e450 2024-08-15T17:58:41.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.378+0000 7fa952d9d700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] conn(0x7fa934068aa0 0x7fa93406aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7fa951d9b700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa934060890 0x7fa934062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7fa952d9d700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] conn(0x7fa934068aa0 0x7fa93406aeb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7fa951d9b700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa934060890 0x7fa934062d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa9541577b0 tx=0x7fa94c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7fa9427fc700 1 -- 172.21.15.38:0/431782136 <== osd.6 v2:172.21.15.104:6808/24497 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa93406b560 con 0x7fa934068aa0 2024-08-15T17:58:41.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f1c07fff700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c080ad750 0x7f1c080afb50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800c5b0 tx=0x7f1bf8007f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f4820e68700 1 -- 172.21.15.38:0/524128191 shutdown_connections 2024-08-15T17:58:41.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f4820e68700 1 --2- 172.21.15.38:0/524128191 >> v2:172.21.15.104:3300/0 conn(0x7f481c0aa980 0x7f481c0b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f4820e68700 1 --2- 172.21.15.38:0/524128191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f481c15a8b0 0x7f481c0aa440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f4820e68700 1 --2- 172.21.15.38:0/524128191 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f481c1593b0 0x7f481c15a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f4820e68700 1 -- 172.21.15.38:0/524128191 >> 172.21.15.38:0/524128191 conn(0x7f481c068300 msgr2=0x7f481c0a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f2feb7dc700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] conn(0x7f2fe4156b80 0x7f2fe4156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f2d6cee5700 1 -- 172.21.15.38:0/776432635 shutdown_connections 2024-08-15T17:58:41.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f2d6cee5700 1 --2- 172.21.15.38:0/776432635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d680b0a90 0x7f2d6806df10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f2d6cee5700 1 --2- 172.21.15.38:0/776432635 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d68157a10 0x7f2d6815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f2d6cee5700 1 -- 172.21.15.38:0/776432635 >> 172.21.15.38:0/776432635 conn(0x7f2d680678d0 msgr2=0x7f2d68067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f1beffff700 1 -- 172.21.15.38:0/3847934128 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8011590 con 0x7f1c080ad750 2024-08-15T17:58:41.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f2feb7dc700 1 -- 172.21.15.38:0/709452479 --> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fe406e590 con 0x7f2fe4156b80 2024-08-15T17:58:41.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f1beffff700 1 -- 172.21.15.38:0/3847934128 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf8011710 con 0x7f1c080ad750 2024-08-15T17:58:41.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f1beffff700 1 -- 172.21.15.38:0/3847934128 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8011ad0 con 0x7f1c080ad750 2024-08-15T17:58:41.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f2fe9d79700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] conn(0x7f2fe4156b80 0x7f2fe4156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f1c0d340700 1 -- 172.21.15.38:0/3847934128 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c080b0090 con 0x7f1c080ad750 2024-08-15T17:58:41.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f4820e68700 1 -- 172.21.15.38:0/524128191 shutdown_connections 2024-08-15T17:58:41.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f1c0d340700 1 -- 172.21.15.38:0/3847934128 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c080b05f0 con 0x7f1c080ad750 2024-08-15T17:58:41.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f2fe9d79700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] conn(0x7f2fe4156b80 0x7f2fe4156f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f2fd97fa700 1 -- 172.21.15.38:0/709452479 <== osd.0 v2:172.21.15.38:6808/24121 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2fe406e590 con 0x7f2fe4156b80 2024-08-15T17:58:41.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f2d6cee5700 1 -- 172.21.15.38:0/776432635 shutdown_connections 2024-08-15T17:58:41.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f4820e68700 1 -- 172.21.15.38:0/524128191 wait complete. 2024-08-15T17:58:41.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f4820e68700 1 Processor -- start 2024-08-15T17:58:41.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f2d6cee5700 1 -- 172.21.15.38:0/776432635 wait complete. 2024-08-15T17:58:41.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f1beffff700 1 -- 172.21.15.38:0/3847934128 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1bf801b3b0 con 0x7f1c080ad750 2024-08-15T17:58:41.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f1beffff700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1bf4060890 0x7f1bf4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f1beffff700 1 -- 172.21.15.38:0/3847934128 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1bf807c020 con 0x7f1c080ad750 2024-08-15T17:58:41.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f1c077fe700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1bf4060890 0x7f1bf4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f4820e68700 1 -- start start 2024-08-15T17:58:41.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d6cee5700 1 Processor -- start 2024-08-15T17:58:41.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f1c077fe700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1bf4060890 0x7f1bf4062d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c08171d50 tx=0x7f1c0000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d6cee5700 1 -- start start 2024-08-15T17:58:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d6cee5700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d6806e450 0x7f2d68139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d6cee5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d680b0a90 0x7f2d681569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d6cee5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d68157a10 0x7f2d6815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d6cee5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6815ab70 con 0x7f2d680b0a90 2024-08-15T17:58:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d6cee5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6816e300 con 0x7f2d68157a10 2024-08-15T17:58:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d6cee5700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6813a530 con 0x7f2d6806e450 2024-08-15T17:58:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f4820e68700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f481c0aa980 0x7f481c097400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f4820e68700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c1593b0 0x7f481c0cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f4820e68700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f481c15a8b0 0x7f481c099160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f4820e68700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c176590 con 0x7f481c15a8b0 2024-08-15T17:58:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f4820e68700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c097a90 con 0x7f481c1593b0 2024-08-15T17:58:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f4820e68700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c097d70 con 0x7f481c0aa980 2024-08-15T17:58:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f481ad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f481c15a8b0 0x7f481c099160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f481ad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f481c15a8b0 0x7f481c099160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53740/0 (socket says 172.21.15.38:53740) 2024-08-15T17:58:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f481ad9d700 1 -- 172.21.15.38:0/259511107 learned_addr learned my addr 172.21.15.38:0/259511107 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d66ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d680b0a90 0x7f2d681569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d68157a10 0x7f2d6815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d66ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d680b0a90 0x7f2d681569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53748/0 (socket says 172.21.15.38:53748) 2024-08-15T17:58:41.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d66ffd700 1 -- 172.21.15.38:0/3841843327 learned_addr learned my addr 172.21.15.38:0/3841843327 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d677fe700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d6806e450 0x7f2d68139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f4819d9b700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c1593b0 0x7f481c0cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f481ad9d700 1 -- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f481c0aa980 msgr2=0x7f481c097400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T17:58:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f481ad9d700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f481c0aa980 0x7f481c097400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f481ad9d700 1 -- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c1593b0 msgr2=0x7f481c0cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f481ad9d700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c1593b0 0x7f481c0cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f481ad9d700 1 -- 172.21.15.38:0/259511107 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4814009ae0 con 0x7f481c15a8b0 2024-08-15T17:58:41.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d67fff700 1 -- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d6806e450 msgr2=0x7f2d68139ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d67fff700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d6806e450 0x7f2d68139ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d67fff700 1 -- 172.21.15.38:0/3841843327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d680b0a90 msgr2=0x7f2d681569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d67fff700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d680b0a90 0x7f2d681569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d67fff700 1 -- 172.21.15.38:0/3841843327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c0094f0 con 0x7f2d68157a10 2024-08-15T17:58:41.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d66ffd700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d680b0a90 0x7f2d681569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:41.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f481ad9d700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f481c15a8b0 0x7f481c099160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f481000b5e0 tx=0x7f4810009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f2d67fff700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d68157a10 0x7f2d6815bf90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800c490 tx=0x7f2d58007ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f575183c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f574c066e00 0x7f574c1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f575183c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f574c0671d0 0x7f574c155d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f575183c700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c1573c0 con 0x7f574c1562a0 2024-08-15T17:58:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f575183c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c156d40 con 0x7f574c0671d0 2024-08-15T17:58:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f575183c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c15c6d0 con 0x7f574c066e00 2024-08-15T17:58:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f574b7fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f574c1562a0 0x7f574c0acb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:59618/0 (socket says 172.21.15.38:59618) 2024-08-15T17:58:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f574b7fe700 1 -- 172.21.15.38:0/1586855805 learned_addr learned my addr 172.21.15.38:0/1586855805 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f574affd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f574c066e00 0x7f574c1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f574a7fc700 1 --2- 172.21.15.38:0/1586855805 >> v2:172.21.15.104:3300/0 conn(0x7f574c0671d0 0x7f574c155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f5749ffb700 1 -- 172.21.15.38:0/1586855805 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211750286 0 0) 0x7f574c1573c0 con 0x7f574c1562a0 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.382+0000 7f5749ffb700 1 -- 172.21.15.38:0/1586855805 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5734003570 con 0x7f574c1562a0 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f574affd700 1 -- 172.21.15.38:0/1586855805 >> v2:172.21.15.104:3300/0 conn(0x7f574c0671d0 msgr2=0x7f574c155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f574affd700 1 --2- 172.21.15.38:0/1586855805 >> v2:172.21.15.104:3300/0 conn(0x7f574c0671d0 0x7f574c155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f574affd700 1 -- 172.21.15.38:0/1586855805 >> v1:172.21.15.104:6789/0 conn(0x7f574c1562a0 legacy=0x7f574c0acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f574affd700 1 -- 172.21.15.38:0/1586855805 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c156780 con 0x7f574c066e00 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f574a7fc700 1 --2- 172.21.15.38:0/1586855805 >> v2:172.21.15.104:3300/0 conn(0x7f574c0671d0 0x7f574c155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f574affd700 1 --2- 172.21.15.38:0/1586855805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f574c066e00 0x7f574c1530c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f573c0098e0 tx=0x7f573c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0aa3f158f80eeb5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f5749ffb700 1 -- 172.21.15.38:0/1586855805 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c011630 con 0x7f574c066e00 2024-08-15T17:58:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f5749ffb700 1 -- 172.21.15.38:0/1586855805 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573c011790 con 0x7f574c066e00 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f5749ffb700 1 -- 172.21.15.38:0/1586855805 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c011b10 con 0x7f574c066e00 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f575183c700 1 -- 172.21.15.38:0/1586855805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f574c066e00 msgr2=0x7f574c1530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f575183c700 1 --2- 172.21.15.38:0/1586855805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f574c066e00 0x7f574c1530c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f573c0098e0 tx=0x7f573c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f575183c700 1 -- 172.21.15.38:0/1586855805 shutdown_connections 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f575183c700 1 --2- 172.21.15.38:0/1586855805 >> v2:172.21.15.104:3300/0 conn(0x7f574c0671d0 0x7f574c155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f575183c700 1 --2- 172.21.15.38:0/1586855805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f574c066e00 0x7f574c1530c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f575183c700 1 -- 172.21.15.38:0/1586855805 >> 172.21.15.38:0/1586855805 conn(0x7f574c0cd160 msgr2=0x7f574c06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f575183c700 1 -- 172.21.15.38:0/1586855805 shutdown_connections 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f575183c700 1 -- 172.21.15.38:0/1586855805 wait complete. 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f575183c700 1 Processor -- start 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.386+0000 7f575183c700 1 -- start start 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f575183c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f574c066e00 0x7f574c1659e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f575183c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c0671d0 0x7f574c06acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f575183c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f574c1562a0 0x7f574c1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f575183c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c165f20 con 0x7f574c066e00 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f575183c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c069f30 con 0x7f574c0671d0 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f575183c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c06a210 con 0x7f574c1562a0 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f574c066e00 0x7f574c1659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574b7fe700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f574c1562a0 0x7f574c1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c0671d0 0x7f574c06acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c0671d0 0x7f574c06acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:35866/0 (socket says 172.21.15.38:35866) 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f574c066e00 0x7f574c1659e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53730/0 (socket says 172.21.15.38:53730) 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574a7fc700 1 -- 172.21.15.38:0/1127099425 learned_addr learned my addr 172.21.15.38:0/1127099425 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574affd700 1 -- 172.21.15.38:0/1127099425 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f574c1562a0 msgr2=0x7f574c1585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574affd700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f574c1562a0 0x7f574c1585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574affd700 1 -- 172.21.15.38:0/1127099425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c0671d0 msgr2=0x7f574c06acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574affd700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c0671d0 0x7f574c06acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574affd700 1 -- 172.21.15.38:0/1127099425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5740009420 con 0x7f574c066e00 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f574affd700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f574c066e00 0x7f574c1659e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f573c016000 tx=0x7f573c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f5732ffd700 1 -- 172.21.15.38:0/1127099425 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c011900 con 0x7f574c066e00 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f5732ffd700 1 -- 172.21.15.38:0/1127099425 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573c011a80 con 0x7f574c066e00 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f575183c700 1 -- 172.21.15.38:0/1127099425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573c009420 con 0x7f574c066e00 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f5732ffd700 1 -- 172.21.15.38:0/1127099425 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c01a680 con 0x7f574c066e00 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.390+0000 7f575183c700 1 -- 172.21.15.38:0/1127099425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c06a7d0 con 0x7f574c066e00 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f5732ffd700 1 -- 172.21.15.38:0/1127099425 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f573c01aab0 con 0x7f574c066e00 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f5732ffd700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5738060860 0x7f5738062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f5732ffd700 1 -- 172.21.15.38:0/1127099425 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f573c07e690 con 0x7f574c066e00 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f574a7fc700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5738060860 0x7f5738062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.394+0000 7f574a7fc700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5738060860 0x7f5738062d10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f574c06b5c0 tx=0x7f5740008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.398+0000 7f575183c700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] conn(0x7f574c0ac8e0 0x7f574c14d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.398+0000 7f575183c700 1 -- 172.21.15.38:0/1127099425 --> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f574c0645c0 con 0x7f574c0ac8e0 2024-08-15T17:58:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.398+0000 7f574b7fe700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] conn(0x7f574c0ac8e0 0x7f574c14d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.398+0000 7f574b7fe700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] conn(0x7f574c0ac8e0 0x7f574c14d530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.398+0000 7f5732ffd700 1 -- 172.21.15.38:0/1127099425 <== osd.3 v2:172.21.15.38:6800/24120 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f574c0645c0 con 0x7f574c0ac8e0 2024-08-15T17:58:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.402+0000 7f1bedffb700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] conn(0x7f1c080b07e0 0x7f1c08170c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.402+0000 7f1bedffb700 1 -- 172.21.15.38:0/3847934128 --> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c080643a0 con 0x7f1c080b07e0 2024-08-15T17:58:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.402+0000 7f1c0cb3f700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] conn(0x7f1c080b07e0 0x7f1c08170c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.402+0000 7f1c0cb3f700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] conn(0x7f1c080b07e0 0x7f1c08170c40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.402+0000 7f1beffff700 1 -- 172.21.15.38:0/3847934128 <== osd.2 v2:172.21.15.38:6824/24123 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1c080643a0 con 0x7f1c080b07e0 2024-08-15T17:58:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.406+0000 7f480a7fc700 1 -- 172.21.15.38:0/259511107 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481000f510 con 0x7f481c15a8b0 2024-08-15T17:58:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.406+0000 7f480a7fc700 1 -- 172.21.15.38:0/259511107 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481000f6b0 con 0x7f481c15a8b0 2024-08-15T17:58:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.406+0000 7f2d4f7fe700 1 -- 172.21.15.38:0/3841843327 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800ce40 con 0x7f2d68157a10 2024-08-15T17:58:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.406+0000 7f2d6cee5700 1 -- 172.21.15.38:0/3841843327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d60009a20 con 0x7f2d68157a10 2024-08-15T17:58:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.406+0000 7f2d6cee5700 1 -- 172.21.15.38:0/3841843327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6813a880 con 0x7f2d68157a10 2024-08-15T17:58:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.406+0000 7f2d4f7fe700 1 -- 172.21.15.38:0/3841843327 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d58011590 con 0x7f2d68157a10 2024-08-15T17:58:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.406+0000 7f2d4f7fe700 1 -- 172.21.15.38:0/3841843327 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58011950 con 0x7f2d68157a10 2024-08-15T17:58:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.406+0000 7f480a7fc700 1 -- 172.21.15.38:0/259511107 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481000fa70 con 0x7f481c15a8b0 2024-08-15T17:58:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.406+0000 7f4820e68700 1 -- 172.21.15.38:0/259511107 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480c009420 con 0x7f481c15a8b0 2024-08-15T17:58:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.406+0000 7fbd63fff700 1 -- 172.21.15.38:0/2348640074 --> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbd44002d30 con 0x7fbd50068a70 2024-08-15T17:58:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd4affd700 1 -- 172.21.15.38:0/2348640074 <== osd.1 v2:172.21.15.38:6816/24122 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbd44002d30 con 0x7fbd50068a70 2024-08-15T17:58:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.406+0000 7f4820e68700 1 -- 172.21.15.38:0/259511107 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c0cb1f0 con 0x7f481c15a8b0 2024-08-15T17:58:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f4820e68700 1 -- 172.21.15.38:0/259511107 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f481c070360 con 0x7f481c15a8b0 2024-08-15T17:58:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f2d4f7fe700 1 -- 172.21.15.38:0/3841843327 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2d58011d40 con 0x7f2d68157a10 2024-08-15T17:58:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f2d6cee5700 1 -- 172.21.15.38:0/3841843327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2d48000fd0 con 0x7f2d68157a10 2024-08-15T17:58:41.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 --> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feeac0b0810 con 0x7feeac153e00 2024-08-15T17:58:41.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd63fff700 1 -- 172.21.15.38:0/2348640074 >> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] conn(0x7fbd50068a70 msgr2=0x7fbd5006ae80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd63fff700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] conn(0x7fbd50068a70 0x7fbd5006ae80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f2d4f7fe700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2d54060930 0x7f2d54062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd63fff700 1 -- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbd50060860 msgr2=0x7fbd50062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd63fff700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbd50060860 0x7fbd50062d10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400adc0 tx=0x7fbd54006f90 comp rx=0 tx=0).stop 2024-08-15T17:58:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd63fff700 1 -- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd5c0e00e0 msgr2=0x7fbd5c0f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd63fff700 1 --2- 172.21.15.38:0/2348640074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd5c0e00e0 0x7fbd5c0f5070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c490 tx=0x7fbd58007e80 comp rx=0 tx=0).stop 2024-08-15T17:58:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa95359e700 1 -- 172.21.15.38:0/431782136 --> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa938002d30 con 0x7fa934068aa0 2024-08-15T17:58:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feea27fc700 1 -- 172.21.15.38:0/2528489537 <== osd.5 v2:172.21.15.104:6800/24498 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7feeac0b0810 con 0x7feeac153e00 2024-08-15T17:58:41.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f2d4f7fe700 1 -- 172.21.15.38:0/3841843327 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2d5807c5d0 con 0x7f2d68157a10 2024-08-15T17:58:41.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd637fe700 1 -- 172.21.15.38:0/2348640074 reap_dead start 2024-08-15T17:58:41.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f2d677fe700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2d54060930 0x7f2d54062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd63fff700 1 -- 172.21.15.38:0/2348640074 shutdown_connections 2024-08-15T17:58:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd63fff700 1 -- 172.21.15.38:0/2348640074 >> 172.21.15.38:0/2348640074 conn(0x7fbd5c00a030 msgr2=0x7fbd5c0db9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f480a7fc700 1 -- 172.21.15.38:0/259511107 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f481001a400 con 0x7f481c15a8b0 2024-08-15T17:58:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd63fff700 1 -- 172.21.15.38:0/2348640074 shutdown_connections 2024-08-15T17:58:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fbd63fff700 1 -- 172.21.15.38:0/2348640074 wait complete. 2024-08-15T17:58:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] conn(0x7feeac153e00 msgr2=0x7feeac155720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb3360700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] conn(0x7feeac153e00 0x7feeac155720 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa9427fc700 1 -- 172.21.15.38:0/431782136 <== osd.6 v2:172.21.15.104:6808/24497 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa938002d30 con 0x7fa934068aa0 2024-08-15T17:58:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f2d4f7fe700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] conn(0x7f2d54068b40 0x7f2d5406af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f2d4f7fe700 1 -- 172.21.15.38:0/3841843327 --> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d5406b600 con 0x7f2d54068b40 2024-08-15T17:58:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f2d4f7fe700 1 -- 172.21.15.38:0/3841843327 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d5807ec10 con 0x7f2d68157a10 2024-08-15T17:58:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f480a7fc700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f47fc060930 0x7f47fc062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f480a7fc700 1 -- 172.21.15.38:0/259511107 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f481007c5a0 con 0x7f481c15a8b0 2024-08-15T17:58:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f480a7fc700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] conn(0x7f47fc068950 0x7f47fc06ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f480a7fc700 1 -- 172.21.15.38:0/259511107 --> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47fc06b410 con 0x7f47fc068950 2024-08-15T17:58:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f2d66ffd700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] conn(0x7f2d54068b40 0x7f2d5406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f481a59c700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f47fc060930 0x7f47fc062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f480a7fc700 1 -- 172.21.15.38:0/259511107 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f481004ca50 con 0x7f481c15a8b0 2024-08-15T17:58:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fee900609b0 msgr2=0x7fee90062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa95359e700 1 -- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] conn(0x7fa934068aa0 msgr2=0x7fa93406aeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa95359e700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] conn(0x7fa934068aa0 0x7fa93406aeb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb3360700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fee900609b0 0x7fee90062e60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feea400f3c0 tx=0x7feea4009040 comp rx=0 tx=0).stop 2024-08-15T17:58:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f2d677fe700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2d54060930 0x7f2d54062de0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c006d20 tx=0x7f2d5c009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7f4819d9b700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] conn(0x7f47fc068950 0x7f47fc06ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeac154e50 msgr2=0x7feeac1455a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb3360700 1 --2- 172.21.15.38:0/2528489537 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeac154e50 0x7feeac1455a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fee9c016000 tx=0x7fee9c017000 comp rx=0 tx=0).stop 2024-08-15T17:58:41.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb2b5f700 1 -- 172.21.15.38:0/2528489537 reap_dead start 2024-08-15T17:58:41.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa95359e700 1 -- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa934060890 msgr2=0x7fa934062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa95359e700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa934060890 0x7fa934062d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa9541577b0 tx=0x7fa94c00b040 comp rx=0 tx=0).stop 2024-08-15T17:58:41.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa95359e700 1 -- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa95406e450 msgr2=0x7fa95415b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa95359e700 1 --2- 172.21.15.38:0/431782136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa95406e450 0x7fa95415b0d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c490 tx=0x7fa944007ea0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa952d9d700 1 -- 172.21.15.38:0/431782136 reap_dead start 2024-08-15T17:58:41.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 shutdown_connections 2024-08-15T17:58:41.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 >> 172.21.15.38:0/2528489537 conn(0x7feeac0678a0 msgr2=0x7feeac06e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa95359e700 1 -- 172.21.15.38:0/431782136 shutdown_connections 2024-08-15T17:58:41.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa95359e700 1 -- 172.21.15.38:0/431782136 >> 172.21.15.38:0/431782136 conn(0x7fa9540678d0 msgr2=0x7fa95406a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 shutdown_connections 2024-08-15T17:58:41.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7feeb3360700 1 -- 172.21.15.38:0/2528489537 wait complete. 2024-08-15T17:58:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.410+0000 7fa95359e700 1 -- 172.21.15.38:0/431782136 shutdown_connections 2024-08-15T17:58:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.414+0000 7fa95359e700 1 -- 172.21.15.38:0/431782136 wait complete. 2024-08-15T17:58:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.414+0000 7f481a59c700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f47fc060930 0x7f47fc062de0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f481400e4a0 tx=0x7f481400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.414+0000 7f4819d9b700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] conn(0x7f47fc068950 0x7f47fc06ad60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.414+0000 7f2d66ffd700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] conn(0x7f2d54068b40 0x7f2d5406af50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.414+0000 7f480a7fc700 1 -- 172.21.15.38:0/259511107 <== osd.7 v2:172.21.15.104:6816/24504 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f47fc06b410 con 0x7f47fc068950 2024-08-15T17:58:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.414+0000 7f2d4f7fe700 1 -- 172.21.15.38:0/3841843327 <== osd.4 v2:172.21.15.104:6824/24496 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2d5406b600 con 0x7f2d54068b40 2024-08-15T17:58:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2feb7dc700 1 -- 172.21.15.38:0/709452479 --> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2fe4157490 con 0x7f2fe4156b80 2024-08-15T17:58:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2fd97fa700 1 -- 172.21.15.38:0/709452479 <== osd.0 v2:172.21.15.38:6808/24121 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2fe4157490 con 0x7f2fe4156b80 2024-08-15T17:58:41.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2feb7dc700 1 -- 172.21.15.38:0/709452479 >> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] conn(0x7f2fe4156b80 msgr2=0x7f2fe4156f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2feb7dc700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] conn(0x7f2fe4156b80 0x7f2fe4156f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2feb7dc700 1 -- 172.21.15.38:0/709452479 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2fc8060860 msgr2=0x7f2fc8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2feb7dc700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2fc8060860 0x7f2fc8062d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009ab0 tx=0x7f2fd4009060 comp rx=0 tx=0).stop 2024-08-15T17:58:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2feb7dc700 1 -- 172.21.15.38:0/709452479 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fe4068be0 msgr2=0x7f2fe40a8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2feb7dc700 1 --2- 172.21.15.38:0/709452479 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fe4068be0 0x7f2fe40a8290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009290 tx=0x7f2fe0016000 comp rx=0 tx=0).stop 2024-08-15T17:58:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2fe9d79700 1 -- 172.21.15.38:0/709452479 reap_dead start 2024-08-15T17:58:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2feb7dc700 1 -- 172.21.15.38:0/709452479 shutdown_connections 2024-08-15T17:58:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2feb7dc700 1 -- 172.21.15.38:0/709452479 >> 172.21.15.38:0/709452479 conn(0x7f2fe406d800 msgr2=0x7f2fe40ab6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2feb7dc700 1 -- 172.21.15.38:0/709452479 shutdown_connections 2024-08-15T17:58:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.418+0000 7f2feb7dc700 1 -- 172.21.15.38:0/709452479 wait complete. 2024-08-15T17:58:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.422+0000 7f575183c700 1 -- 172.21.15.38:0/1127099425 --> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f574c06e530 con 0x7f574c0ac8e0 2024-08-15T17:58:41.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.426+0000 7f5732ffd700 1 -- 172.21.15.38:0/1127099425 <== osd.3 v2:172.21.15.38:6800/24120 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f574c06e530 con 0x7f574c0ac8e0 2024-08-15T17:58:41.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f1bedffb700 1 -- 172.21.15.38:0/3847934128 --> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c081563d0 con 0x7f1c080b07e0 2024-08-15T17:58:41.460 INFO:teuthology.orchestra.run.smithi038.stdout:60129542150 2024-08-15T17:58:41.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f575183c700 1 -- 172.21.15.38:0/1127099425 >> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] conn(0x7f574c0ac8e0 msgr2=0x7f574c14d530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f575183c700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] conn(0x7f574c0ac8e0 0x7f574c14d530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.461 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-15T17:58:41.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f575183c700 1 -- 172.21.15.38:0/1127099425 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5738060860 msgr2=0x7f5738062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f575183c700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5738060860 0x7f5738062d10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f574c06b5c0 tx=0x7f5740008040 comp rx=0 tx=0).stop 2024-08-15T17:58:41.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f575183c700 1 -- 172.21.15.38:0/1127099425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f574c066e00 msgr2=0x7f574c1659e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f575183c700 1 --2- 172.21.15.38:0/1127099425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f574c066e00 0x7f574c1659e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f573c016000 tx=0x7f573c017000 comp rx=0 tx=0).stop 2024-08-15T17:58:41.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f1beffff700 1 -- 172.21.15.38:0/3847934128 <== osd.2 v2:172.21.15.38:6824/24123 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1c081563d0 con 0x7f1c080b07e0 2024-08-15T17:58:41.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f574b7fe700 1 -- 172.21.15.38:0/1127099425 reap_dead start 2024-08-15T17:58:41.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f575183c700 1 -- 172.21.15.38:0/1127099425 shutdown_connections 2024-08-15T17:58:41.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f575183c700 1 -- 172.21.15.38:0/1127099425 >> 172.21.15.38:0/1127099425 conn(0x7f574c0cd160 msgr2=0x7f574c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f575183c700 1 -- 172.21.15.38:0/1127099425 shutdown_connections 2024-08-15T17:58:41.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.430+0000 7f575183c700 1 -- 172.21.15.38:0/1127099425 wait complete. 2024-08-15T17:58:41.464 INFO:teuthology.orchestra.run.smithi038.stdout:64424509447 2024-08-15T17:58:41.464 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-15T17:58:41.465 INFO:teuthology.orchestra.run.smithi038.stdout:60129542151 2024-08-15T17:58:41.466 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-15T17:58:41.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.434+0000 7f1bedffb700 1 -- 172.21.15.38:0/3847934128 >> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] conn(0x7f1c080b07e0 msgr2=0x7f1c08170c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.434+0000 7f1bedffb700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] conn(0x7f1c080b07e0 0x7f1c08170c40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.434+0000 7f1bedffb700 1 -- 172.21.15.38:0/3847934128 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1bf4060890 msgr2=0x7f1bf4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.434+0000 7f1bedffb700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1bf4060890 0x7f1bf4062d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c08171d50 tx=0x7f1c0000b040 comp rx=0 tx=0).stop 2024-08-15T17:58:41.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.434+0000 7f1bedffb700 1 -- 172.21.15.38:0/3847934128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c080ad750 msgr2=0x7f1c080afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.434+0000 7f1bedffb700 1 --2- 172.21.15.38:0/3847934128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c080ad750 0x7f1c080afb50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800c5b0 tx=0x7f1bf8007f60 comp rx=0 tx=0).stop 2024-08-15T17:58:41.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.434+0000 7f1c0cb3f700 1 -- 172.21.15.38:0/3847934128 reap_dead start 2024-08-15T17:58:41.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.434+0000 7f1bedffb700 1 -- 172.21.15.38:0/3847934128 shutdown_connections 2024-08-15T17:58:41.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.434+0000 7f1bedffb700 1 -- 172.21.15.38:0/3847934128 >> 172.21.15.38:0/3847934128 conn(0x7f1c080678d0 msgr2=0x7f1c08069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.438+0000 7f1bedffb700 1 -- 172.21.15.38:0/3847934128 shutdown_connections 2024-08-15T17:58:41.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.438+0000 7f1bedffb700 1 -- 172.21.15.38:0/3847934128 wait complete. 2024-08-15T17:58:41.468 INFO:teuthology.orchestra.run.smithi038.stdout:60129542151 2024-08-15T17:58:41.468 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-15T17:58:41.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.438+0000 7f2d6cee5700 1 -- 172.21.15.38:0/3841843327 --> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2d48002d30 con 0x7f2d54068b40 2024-08-15T17:58:41.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.438+0000 7f2d4f7fe700 1 -- 172.21.15.38:0/3841843327 <== osd.4 v2:172.21.15.104:6824/24496 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2d48002d30 con 0x7f2d54068b40 2024-08-15T17:58:41.470 INFO:teuthology.orchestra.run.smithi038.stdout:60129542152 2024-08-15T17:58:41.470 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-15T17:58:41.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f2d6cee5700 1 -- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] conn(0x7f2d54068b40 msgr2=0x7f2d5406af50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f2d6cee5700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] conn(0x7f2d54068b40 0x7f2d5406af50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f4820e68700 1 -- 172.21.15.38:0/259511107 --> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f481c0645c0 con 0x7f47fc068950 2024-08-15T17:58:41.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f2d6cee5700 1 -- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2d54060930 msgr2=0x7f2d54062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f2d6cee5700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2d54060930 0x7f2d54062de0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c006d20 tx=0x7f2d5c009250 comp rx=0 tx=0).stop 2024-08-15T17:58:41.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f2d6cee5700 1 -- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d68157a10 msgr2=0x7f2d6815bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f2d6cee5700 1 --2- 172.21.15.38:0/3841843327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d68157a10 0x7f2d6815bf90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800c490 tx=0x7f2d58007ea0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f2d67fff700 1 -- 172.21.15.38:0/3841843327 reap_dead start 2024-08-15T17:58:41.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f2d6cee5700 1 -- 172.21.15.38:0/3841843327 shutdown_connections 2024-08-15T17:58:41.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f2d6cee5700 1 -- 172.21.15.38:0/3841843327 >> 172.21.15.38:0/3841843327 conn(0x7f2d680678d0 msgr2=0x7f2d6806cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f2d6cee5700 1 -- 172.21.15.38:0/3841843327 shutdown_connections 2024-08-15T17:58:41.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f2d6cee5700 1 -- 172.21.15.38:0/3841843327 wait complete. 2024-08-15T17:58:41.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f480a7fc700 1 -- 172.21.15.38:0/259511107 <== osd.7 v2:172.21.15.104:6816/24504 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f481c0645c0 con 0x7f47fc068950 2024-08-15T17:58:41.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f4820e68700 1 -- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] conn(0x7f47fc068950 msgr2=0x7f47fc06ad60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f4820e68700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] conn(0x7f47fc068950 0x7f47fc06ad60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f4820e68700 1 -- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f47fc060930 msgr2=0x7f47fc062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f4820e68700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f47fc060930 0x7f47fc062de0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f481400e4a0 tx=0x7f481400b040 comp rx=0 tx=0).stop 2024-08-15T17:58:41.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f4820e68700 1 -- 172.21.15.38:0/259511107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f481c15a8b0 msgr2=0x7f481c099160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f4820e68700 1 --2- 172.21.15.38:0/259511107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f481c15a8b0 0x7f481c099160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f481000b5e0 tx=0x7f4810009120 comp rx=0 tx=0).stop 2024-08-15T17:58:41.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f481ad9d700 1 -- 172.21.15.38:0/259511107 reap_dead start 2024-08-15T17:58:41.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f4820e68700 1 -- 172.21.15.38:0/259511107 shutdown_connections 2024-08-15T17:58:41.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f4820e68700 1 -- 172.21.15.38:0/259511107 >> 172.21.15.38:0/259511107 conn(0x7f481c068300 msgr2=0x7f481c0b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f4820e68700 1 -- 172.21.15.38:0/259511107 shutdown_connections 2024-08-15T17:58:41.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.442+0000 7f4820e68700 1 -- 172.21.15.38:0/259511107 wait complete. 2024-08-15T17:58:41.475 INFO:teuthology.orchestra.run.smithi038.stdout:60129542150 2024-08-15T17:58:41.476 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-15T17:58:41.476 INFO:teuthology.orchestra.run.smithi038.stdout:64424509447 2024-08-15T17:58:41.477 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-15T17:58:41.478 INFO:teuthology.orchestra.run.smithi038.stdout:64424509448 2024-08-15T17:58:41.478 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-15T17:58:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.658+0000 7f7838989700 1 Processor -- start 2024-08-15T17:58:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.658+0000 7fb313793700 1 Processor -- start 2024-08-15T17:58:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.658+0000 7fb313793700 1 -- start start 2024-08-15T17:58:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb313793700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb30c158f00 0x7fb30c0b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb313793700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb30c0b4220 0x7fb30c0b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb313793700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30c0b6020 con 0x7fb30c0b0aa0 2024-08-15T17:58:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb313793700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb30c0b1020 con 0x7fb30c158f00 2024-08-15T17:58:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb313793700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb30c0b1300 con 0x7fb30c0b4220 2024-08-15T17:58:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.658+0000 7f7838989700 1 -- start start 2024-08-15T17:58:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7838989700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f782c0081e0 0x7f782c0396f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7838989700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c0dfee0 con 0x7f782c007e10 2024-08-15T17:58:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7838989700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c0e0130 con 0x7f782c056920 2024-08-15T17:58:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7838989700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782c04ac40 con 0x7f782c0081e0 2024-08-15T17:58:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb310d2e700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb30c0b0aa0 0x7fb30c0b3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40124/0 (socket says 172.21.15.38:40124) 2024-08-15T17:58:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb310d2e700 1 -- 172.21.15.38:0/2816603342 learned_addr learned my addr 172.21.15.38:0/2816603342 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783259c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f782c007e10 0x7f782c056210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40134/0 (socket says 172.21.15.38:40134) 2024-08-15T17:58:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783259c700 1 -- 172.21.15.38:0/2134433393 learned_addr learned my addr 172.21.15.38:0/2134433393 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7832d9d700 1 --2- 172.21.15.38:0/2134433393 >> v2:172.21.15.104:3300/0 conn(0x7f782c0081e0 0x7f782c0396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb311d30700 1 --2- 172.21.15.38:0/2816603342 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb30c0b4220 0x7fb30c0b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb303fff700 1 -- 172.21.15.38:0/2816603342 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553883557 0 0) 0x7fb30c0b6020 con 0x7fb30c0b0aa0 2024-08-15T17:58:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb303fff700 1 -- 172.21.15.38:0/2816603342 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2f8003570 con 0x7fb30c0b0aa0 2024-08-15T17:58:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323720012 0 0) 0x7f782c0dfee0 con 0x7f782c007e10 2024-08-15T17:58:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f781c003570 con 0x7f782c007e10 2024-08-15T17:58:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e41c4a700 1 Processor -- start 2024-08-15T17:58:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517108560 0 0) 0x7f782c0e0130 con 0x7f782c056920 2024-08-15T17:58:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb32b25f700 1 Processor -- start 2024-08-15T17:58:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e41c4a700 1 -- start start 2024-08-15T17:58:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e41c4a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e3c0af190 0x7f7e3c165700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e41c4a700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c166dc0 con 0x7f7e3c06e1a0 2024-08-15T17:58:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e41c4a700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c166490 con 0x7f7e3c06e570 2024-08-15T17:58:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e41c4a700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c166770 con 0x7f7e3c0af190 2024-08-15T17:58:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f782c0dfee0 con 0x7f782c056920 2024-08-15T17:58:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 157838297 0 0) 0x7f781c003570 con 0x7f782c007e10 2024-08-15T17:58:41.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb31152f700 1 --2- 172.21.15.38:0/2816603342 >> v2:172.21.15.38:3300/0 conn(0x7fb30c158f00 0x7fb30c0b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 >> v2:172.21.15.104:3300/0 conn(0x7f782c0081e0 msgr2=0x7f782c0396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 --2- 172.21.15.38:0/2134433393 >> v2:172.21.15.104:3300/0 conn(0x7f782c0081e0 0x7f782c0396f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e41449700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f7e3c06e570 0x7f7e3c165c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:59632/0 (socket says 172.21.15.38:59632) 2024-08-15T17:58:41.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 >> v1:172.21.15.104:6789/0 conn(0x7f782c056920 legacy=0x7f782c052900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb31152f700 1 -- 172.21.15.38:0/2816603342 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb30c0b4220 msgr2=0x7fb30c0b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb303fff700 1 -- 172.21.15.38:0/2816603342 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3383172912 0 0) 0x7fb2f8003570 con 0x7fb30c0b0aa0 2024-08-15T17:58:41.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb31152f700 1 --2- 172.21.15.38:0/2816603342 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb30c0b4220 0x7fb30c0b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb31152f700 1 -- 172.21.15.38:0/2816603342 >> v1:172.21.15.38:6789/0 conn(0x7fb30c0b0aa0 legacy=0x7fb30c0b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb31152f700 1 -- 172.21.15.38:0/2816603342 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c0b15e0 con 0x7fb30c158f00 2024-08-15T17:58:41.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb311d30700 1 --2- 172.21.15.38:0/2816603342 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb30c0b4220 0x7fb30c0b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:41.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e41449700 1 -- 172.21.15.38:0/250242431 learned_addr learned my addr 172.21.15.38:0/250242431 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:41.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e3bfff700 1 --2- 172.21.15.38:0/250242431 >> v2:172.21.15.38:3300/0 conn(0x7f7e3c0af190 0x7f7e3c165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c04af20 con 0x7f782c007e10 2024-08-15T17:58:41.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb32b25f700 1 -- start start 2024-08-15T17:58:41.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f78240014c0 con 0x7f782c007e10 2024-08-15T17:58:41.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 >> v1:172.21.15.38:6789/0 conn(0x7f782c007e10 legacy=0x7f782c056210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f781c003bf0 0x7f781c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e3b7fe700 1 -- 172.21.15.38:0/250242431 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 860772811 0 0) 0x7f7e3c166dc0 con 0x7f7e3c06e1a0 2024-08-15T17:58:41.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e3b7fe700 1 -- 172.21.15.38:0/250242431 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e24003570 con 0x7f7e3c06e1a0 2024-08-15T17:58:41.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e3b7fe700 1 -- 172.21.15.38:0/250242431 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517408963 0 0) 0x7f7e3c166490 con 0x7f7e3c06e570 2024-08-15T17:58:41.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783159a700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f781c004810 0x7f781c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e3b7fe700 1 -- 172.21.15.38:0/250242431 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e3c166dc0 con 0x7f7e3c06e570 2024-08-15T17:58:41.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e3bfff700 1 -- 172.21.15.38:0/250242431 >> v1:172.21.15.104:6789/0 conn(0x7f7e3c06e570 legacy=0x7f7e3c165c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e3bfff700 1 -- 172.21.15.38:0/250242431 >> v1:172.21.15.38:6789/0 conn(0x7f7e3c06e1a0 legacy=0x7f7e3c0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb31152f700 1 --2- 172.21.15.38:0/2816603342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb30c158f00 0x7fb30c0b0560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb308009e60 tx=0x7fb308009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e117de3a4ad788 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb32b25f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb3240abe50 0x7fb3240ae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f783259c700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f781c003bf0 0x7f781c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb32b25f700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb324166980 0x7fb3240b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb32b25f700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb324167bd0 con 0x7fb3240ac220 2024-08-15T17:58:41.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb32b25f700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb324166e70 con 0x7fb324166980 2024-08-15T17:58:41.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb32b25f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb324167150 con 0x7fb3240abe50 2024-08-15T17:58:41.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb3297fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb324166980 0x7fb3240b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb303fff700 1 -- 172.21.15.38:0/2816603342 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30800e070 con 0x7fb30c158f00 2024-08-15T17:58:41.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb303fff700 1 -- 172.21.15.38:0/2816603342 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30800b7a0 con 0x7fb30c158f00 2024-08-15T17:58:41.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb303fff700 1 -- 172.21.15.38:0/2816603342 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc001470 con 0x7fb30c158f00 2024-08-15T17:58:41.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783159a700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f781c0051c0 0x7f781c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c04ac40 con 0x7f781c0051c0 2024-08-15T17:58:41.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c00c050 con 0x7f781c003bf0 2024-08-15T17:58:41.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7f7e3bfff700 1 -- 172.21.15.38:0/250242431 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c166a50 con 0x7f7e3c0af190 2024-08-15T17:58:41.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e3bfff700 1 --2- 172.21.15.38:0/250242431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e3c0af190 0x7f7e3c165700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7e300098e0 tx=0x7f7e300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2618b90629d4381c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c00c210 con 0x7f781c004810 2024-08-15T17:58:41.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7831d9b700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f781c004810 0x7f781c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7832d9d700 1 --2- 172.21.15.38:0/2134433393 >> v2:172.21.15.104:3300/0 conn(0x7f782c0081e0 0x7f782c0396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:41.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e3b7fe700 1 -- 172.21.15.38:0/250242431 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e30011630 con 0x7f7e3c0af190 2024-08-15T17:58:41.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7832d9d700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f781c0051c0 0x7f781c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb3297fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb324166980 0x7fb3240b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53770/0 (socket says 172.21.15.38:53770) 2024-08-15T17:58:41.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.662+0000 7fb3297fc700 1 -- 172.21.15.38:0/1827120310 learned_addr learned my addr 172.21.15.38:0/1827120310 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb328ffb700 1 --2- 172.21.15.38:0/1827120310 >> v2:172.21.15.104:3300/0 conn(0x7fb3240abe50 0x7fb3240ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb3297fc700 1 -- 172.21.15.38:0/1827120310 >> v2:172.21.15.104:3300/0 conn(0x7fb3240abe50 msgr2=0x7fb3240ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb3297fc700 1 --2- 172.21.15.38:0/1827120310 >> v2:172.21.15.104:3300/0 conn(0x7fb3240abe50 0x7fb3240ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb3297fc700 1 -- 172.21.15.38:0/1827120310 >> v1:172.21.15.104:6789/0 conn(0x7fb3240ac220 legacy=0x7fb324166270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb3297fc700 1 -- 172.21.15.38:0/1827120310 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb324167430 con 0x7fb324166980 2024-08-15T17:58:41.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb313793700 1 -- 172.21.15.38:0/2816603342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb30c158f00 msgr2=0x7fb30c0b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb313793700 1 --2- 172.21.15.38:0/2816603342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb30c158f00 0x7fb30c0b0560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb308009e60 tx=0x7fb308009780 comp rx=0 tx=0).stop 2024-08-15T17:58:41.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb3297fc700 1 --2- 172.21.15.38:0/1827120310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb324166980 0x7fb3240b0d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb32406c9d0 tx=0x7fb314009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f2d4f1332a9dca server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e3b7fe700 1 -- 172.21.15.38:0/250242431 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e30011790 con 0x7f7e3c0af190 2024-08-15T17:58:41.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e3b7fe700 1 -- 172.21.15.38:0/250242431 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e30011b30 con 0x7f7e3c0af190 2024-08-15T17:58:41.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb313793700 1 -- 172.21.15.38:0/2816603342 shutdown_connections 2024-08-15T17:58:41.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb313793700 1 --2- 172.21.15.38:0/2816603342 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb30c0b4220 0x7fb30c0b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb313793700 1 --2- 172.21.15.38:0/2816603342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb30c158f00 0x7fb30c0b0560 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb313793700 1 -- 172.21.15.38:0/2816603342 >> 172.21.15.38:0/2816603342 conn(0x7fb30c099510 msgr2=0x7fb30c067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e41c4a700 1 -- 172.21.15.38:0/250242431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e3c0af190 msgr2=0x7f7e3c165700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783259c700 1 -- 172.21.15.38:0/2134433393 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f781c004810 msgr2=0x7f781c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783259c700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f781c004810 0x7f781c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783259c700 1 -- 172.21.15.38:0/2134433393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f781c0051c0 msgr2=0x7f781c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783259c700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f781c0051c0 0x7f781c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e41c4a700 1 --2- 172.21.15.38:0/250242431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e3c0af190 0x7f7e3c165700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7e300098e0 tx=0x7f7e300091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783259c700 1 -- 172.21.15.38:0/2134433393 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c003570 con 0x7f781c003bf0 2024-08-15T17:58:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e41c4a700 1 -- 172.21.15.38:0/250242431 shutdown_connections 2024-08-15T17:58:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e41c4a700 1 --2- 172.21.15.38:0/250242431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e3c0af190 0x7f7e3c165700 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e41c4a700 1 -- 172.21.15.38:0/250242431 >> 172.21.15.38:0/250242431 conn(0x7f7e3c0678d0 msgr2=0x7f7e3c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb313793700 1 -- 172.21.15.38:0/2816603342 shutdown_connections 2024-08-15T17:58:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7832d9d700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f781c0051c0 0x7f781c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7832d9d700 1 -- 172.21.15.38:0/2134433393 reap_dead start 2024-08-15T17:58:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb3237fe700 1 -- 172.21.15.38:0/1827120310 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314009e70 con 0x7fb324166980 2024-08-15T17:58:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb3237fe700 1 -- 172.21.15.38:0/1827120310 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31400b760 con 0x7fb324166980 2024-08-15T17:58:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb313793700 1 -- 172.21.15.38:0/2816603342 wait complete. 2024-08-15T17:58:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783259c700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f781c003bf0 0x7f781c003fc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7824004bc0 tx=0x7f7824008c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7767bdf49a32c8e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e41c4a700 1 -- 172.21.15.38:0/250242431 shutdown_connections 2024-08-15T17:58:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb3237fe700 1 -- 172.21.15.38:0/1827120310 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31400bae0 con 0x7fb324166980 2024-08-15T17:58:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e41c4a700 1 -- 172.21.15.38:0/250242431 wait complete. 2024-08-15T17:58:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824005920 con 0x7f781c003bf0 2024-08-15T17:58:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7824016070 con 0x7f781c003bf0 2024-08-15T17:58:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f783159a700 1 -- 172.21.15.38:0/2134433393 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824005cf0 con 0x7f781c003bf0 2024-08-15T17:58:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb32b25f700 1 -- 172.21.15.38:0/1827120310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb324166980 msgr2=0x7fb3240b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb32b25f700 1 --2- 172.21.15.38:0/1827120310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb324166980 0x7fb3240b0d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb32406c9d0 tx=0x7fb314009250 comp rx=0 tx=0).stop 2024-08-15T17:58:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7838989700 1 -- 172.21.15.38:0/2134433393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f781c003bf0 msgr2=0x7f781c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7838989700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f781c003bf0 0x7f781c003fc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7824004bc0 tx=0x7f7824008c30 comp rx=0 tx=0).stop 2024-08-15T17:58:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb32b25f700 1 -- 172.21.15.38:0/1827120310 shutdown_connections 2024-08-15T17:58:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb32b25f700 1 --2- 172.21.15.38:0/1827120310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb324166980 0x7fb3240b0d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb32b25f700 1 --2- 172.21.15.38:0/1827120310 >> v2:172.21.15.104:3300/0 conn(0x7fb3240abe50 0x7fb3240ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb32b25f700 1 -- 172.21.15.38:0/1827120310 >> 172.21.15.38:0/1827120310 conn(0x7fb324068300 msgr2=0x7fb3240a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7838989700 1 -- 172.21.15.38:0/2134433393 shutdown_connections 2024-08-15T17:58:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7838989700 1 --2- 172.21.15.38:0/2134433393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f781c003bf0 0x7f781c003fc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7838989700 1 -- 172.21.15.38:0/2134433393 >> 172.21.15.38:0/2134433393 conn(0x7f782c038be0 msgr2=0x7f782c038fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7838989700 1 -- 172.21.15.38:0/2134433393 shutdown_connections 2024-08-15T17:58:41.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb32b25f700 1 -- 172.21.15.38:0/1827120310 shutdown_connections 2024-08-15T17:58:41.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7838989700 1 -- 172.21.15.38:0/2134433393 wait complete. 2024-08-15T17:58:41.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb32b25f700 1 -- 172.21.15.38:0/1827120310 wait complete. 2024-08-15T17:58:41.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb313793700 1 Processor -- start 2024-08-15T17:58:41.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7f7e41c4a700 1 Processor -- start 2024-08-15T17:58:41.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.666+0000 7fb313793700 1 -- start start 2024-08-15T17:58:41.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41c4a700 1 -- start start 2024-08-15T17:58:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb32b25f700 1 Processor -- start 2024-08-15T17:58:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7838989700 1 Processor -- start 2024-08-15T17:58:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb32b25f700 1 -- start start 2024-08-15T17:58:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7838989700 1 -- start start 2024-08-15T17:58:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb313793700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb30c0b0aa0 0x7fb30c0ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb313793700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb30c0b4220 0x7fb30c0af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb313793700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb30c158f00 0x7fb30c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb313793700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30c0ac7e0 con 0x7fb30c0b4220 2024-08-15T17:58:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb313793700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30c0abbd0 con 0x7fb30c0b0aa0 2024-08-15T17:58:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb313793700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb30c0abeb0 con 0x7fb30c158f00 2024-08-15T17:58:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41c4a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7e3c06e1a0 0x7f7e3c14dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb31152f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb30c0b0aa0 0x7fb30c0ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb31152f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb30c0b0aa0 0x7fb30c0ab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:35898/0 (socket says 172.21.15.38:35898) 2024-08-15T17:58:41.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb31152f700 1 -- 172.21.15.38:0/3313102658 learned_addr learned my addr 172.21.15.38:0/3313102658 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb310d2e700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb30c0b4220 0x7fb30c0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41c4a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e3c06e570 0x7f7e3c14ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41c4a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e3c0af190 0x7f7e3c0869b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41c4a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c1489f0 con 0x7f7e3c0af190 2024-08-15T17:58:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41c4a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c14b1a0 con 0x7f7e3c06e570 2024-08-15T17:58:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41c4a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c1483e0 con 0x7f7e3c06e1a0 2024-08-15T17:58:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e40c48700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7e3c06e1a0 0x7f7e3c14dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e3bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e3c06e570 0x7f7e3c14ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e3bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e3c06e570 0x7f7e3c14ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:35912/0 (socket says 172.21.15.38:35912) 2024-08-15T17:58:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e3bfff700 1 -- 172.21.15.38:0/2414091301 learned_addr learned my addr 172.21.15.38:0/2414091301 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41449700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e3c0af190 0x7f7e3c0869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb310d2e700 1 -- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb30c158f00 msgr2=0x7fb30c0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb310d2e700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb30c158f00 0x7fb30c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb310d2e700 1 -- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb30c0b0aa0 msgr2=0x7fb30c0ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb310d2e700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb30c0b0aa0 0x7fb30c0ab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb310d2e700 1 -- 172.21.15.38:0/3313102658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb304009420 con 0x7fb30c0b4220 2024-08-15T17:58:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb310d2e700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb30c0b4220 0x7fb30c0af540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00c7e0 tx=0x7fb2fc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7838989700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f781c003bf0 0x7f782c0d69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7838989700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f782c0d7110 0x7f782c0dd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7838989700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782c0d7580 0x7f782c0dd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7838989700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0db600 con 0x7f782c0d7580 2024-08-15T17:58:41.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7838989700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0d6f00 con 0x7f781c003bf0 2024-08-15T17:58:41.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7838989700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0da9f0 con 0x7f782c0d7110 2024-08-15T17:58:41.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb32b25f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3240abe50 0x7fb3240aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb32b25f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb3240ac220 0x7fb3240aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb32b25f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb324166980 0x7fb3241591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb32b25f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240b0920 con 0x7fb3240abe50 2024-08-15T17:58:41.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb32b25f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240af260 con 0x7fb324166980 2024-08-15T17:58:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb32b25f700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240b05d0 con 0x7fb3240ac220 2024-08-15T17:58:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7832d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782c0d7580 0x7f782c0dd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7832d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782c0d7580 0x7f782c0dd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53802/0 (socket says 172.21.15.38:53802) 2024-08-15T17:58:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7832d9d700 1 -- 172.21.15.38:0/761899318 learned_addr learned my addr 172.21.15.38:0/761899318 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41449700 1 -- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7e3c06e1a0 msgr2=0x7f7e3c14dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41449700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7e3c06e1a0 0x7f7e3c14dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41449700 1 -- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e3c06e570 msgr2=0x7f7e3c14ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41449700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e3c06e570 0x7f7e3c14ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41449700 1 -- 172.21.15.38:0/2414091301 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e30009420 con 0x7f7e3c0af190 2024-08-15T17:58:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fdc8a293700 1 Processor -- start 2024-08-15T17:58:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41449700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e3c0af190 0x7f7e3c0869b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00c560 tx=0x7f7e2c007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb323fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb3240ac220 0x7fb3240aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f783259c700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f781c003bf0 0x7f782c0d69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb3017fa700 1 -- 172.21.15.38:0/3313102658 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc007360 con 0x7fb30c0b4220 2024-08-15T17:58:41.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb3017fa700 1 -- 172.21.15.38:0/3313102658 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc0074e0 con 0x7fb30c0b4220 2024-08-15T17:58:41.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb3017fa700 1 -- 172.21.15.38:0/3313102658 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc004830 con 0x7fb30c0b4220 2024-08-15T17:58:41.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb323fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb3240ac220 0x7fb3240aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:44112/0 (socket says 172.21.15.38:44112) 2024-08-15T17:58:41.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb323fff700 1 -- 172.21.15.38:0/2705449349 learned_addr learned my addr 172.21.15.38:0/2705449349 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb313793700 1 -- 172.21.15.38:0/3313102658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb308009a20 con 0x7fb30c0b4220 2024-08-15T17:58:41.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb313793700 1 -- 172.21.15.38:0/3313102658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb30c0ac470 con 0x7fb30c0b4220 2024-08-15T17:58:41.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb3297fc700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb324166980 0x7fb3241591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e38ff9700 1 -- 172.21.15.38:0/2414091301 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c005b30 con 0x7f7e3c0af190 2024-08-15T17:58:41.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f42fac3c700 1 Processor -- start 2024-08-15T17:58:41.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb328ffb700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3240abe50 0x7fb3240aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e38ff9700 1 -- 172.21.15.38:0/2414091301 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c005cb0 con 0x7f7e3c0af190 2024-08-15T17:58:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e38ff9700 1 -- 172.21.15.38:0/2414091301 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c007320 con 0x7f7e3c0af190 2024-08-15T17:58:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41c4a700 1 -- 172.21.15.38:0/2414091301 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c14b400 con 0x7f7e3c0af190 2024-08-15T17:58:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7e41c4a700 1 -- 172.21.15.38:0/2414091301 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c14b6e0 con 0x7f7e3c0af190 2024-08-15T17:58:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7832d9d700 1 -- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f782c0d7110 msgr2=0x7f782c0dd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7832d9d700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f782c0d7110 0x7f782c0dd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7832d9d700 1 -- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f781c003bf0 msgr2=0x7f782c0d69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7832d9d700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f781c003bf0 0x7f782c0d69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7832d9d700 1 -- 172.21.15.38:0/761899318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c0dacd0 con 0x7f782c0d7580 2024-08-15T17:58:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb328ffb700 1 -- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb3240ac220 msgr2=0x7fb3240aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb328ffb700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb3240ac220 0x7fb3240aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb328ffb700 1 -- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb324166980 msgr2=0x7fb3241591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb328ffb700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb324166980 0x7fb3241591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fb328ffb700 1 -- 172.21.15.38:0/2705449349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140094f0 con 0x7fb3240abe50 2024-08-15T17:58:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7f7832d9d700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782c0d7580 0x7f782c0dd550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f782c00dc20 tx=0x7f78280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fb328ffb700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3240abe50 0x7fb3240aa0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb318017180 tx=0x7fb318008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.670+0000 7fdc8a293700 1 -- start start 2024-08-15T17:58:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc8a293700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdc840aae80 0x7fdc8406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc8a293700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc840ab250 0x7fdc8406e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc8a293700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8406ed90 con 0x7fdc84161d50 2024-08-15T17:58:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc8a293700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8406e3c0 con 0x7fdc840aae80 2024-08-15T17:58:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc8a293700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8406d7b0 con 0x7fdc840ab250 2024-08-15T17:58:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc89291700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdc840aae80 0x7fdc8406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc89291700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdc840aae80 0x7fdc8406d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:35948/0 (socket says 172.21.15.38:35948) 2024-08-15T17:58:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc89291700 1 -- 172.21.15.38:0/580426609 learned_addr learned my addr 172.21.15.38:0/580426609 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc88a90700 1 --2- 172.21.15.38:0/580426609 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc840ab250 0x7fdc8406e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f7e41c4a700 1 -- 172.21.15.38:0/2414091301 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c148b60 con 0x7f7e3c0af190 2024-08-15T17:58:41.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42fac3c700 1 -- start start 2024-08-15T17:58:41.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc88a90700 1 -- 172.21.15.38:0/580426609 >> v2:172.21.15.104:3300/0 conn(0x7fdc840aae80 msgr2=0x7fdc8406d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc88a90700 1 --2- 172.21.15.38:0/580426609 >> v2:172.21.15.104:3300/0 conn(0x7fdc840aae80 0x7fdc8406d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc88a90700 1 -- 172.21.15.38:0/580426609 >> v1:172.21.15.38:6789/0 conn(0x7fdc84161d50 legacy=0x7fdc840af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc88a90700 1 -- 172.21.15.38:0/580426609 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8406da90 con 0x7fdc840ab250 2024-08-15T17:58:41.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc88a90700 1 --2- 172.21.15.38:0/580426609 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc840ab250 0x7fdc8406e850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc740098e0 tx=0x7fdc740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91841c02aa87c107 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42fac3c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f42f40ab540 0x7f42f406eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42fac3c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f42f4155720 0x7f42f40b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42fac3c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f40aba70 con 0x7f42f406e9d0 2024-08-15T17:58:41.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42fac3c700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f40abd50 con 0x7f42f40ab540 2024-08-15T17:58:41.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42fac3c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f40ac030 con 0x7f42f4155720 2024-08-15T17:58:41.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42f9439700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f42f40ab540 0x7f42f406eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42f9439700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f42f40ab540 0x7f42f406eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53824/0 (socket says 172.21.15.38:53824) 2024-08-15T17:58:41.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42f9c3a700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f42f406e9d0 0x7f42f4155010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40162/0 (socket says 172.21.15.38:40162) 2024-08-15T17:58:41.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42f9439700 1 -- 172.21.15.38:0/3595831465 learned_addr learned my addr 172.21.15.38:0/3595831465 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f78227fc700 1 -- 172.21.15.38:0/761899318 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782800fc40 con 0x7f782c0d7580 2024-08-15T17:58:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f78227fc700 1 -- 172.21.15.38:0/761899318 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782800fda0 con 0x7f782c0d7580 2024-08-15T17:58:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f78227fc700 1 -- 172.21.15.38:0/761899318 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828014670 con 0x7f782c0d7580 2024-08-15T17:58:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f7838989700 1 -- 172.21.15.38:0/761899318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782c0dafb0 con 0x7f782c0d7580 2024-08-15T17:58:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f7838989700 1 -- 172.21.15.38:0/761899318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c0dba90 con 0x7f782c0d7580 2024-08-15T17:58:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fb320ff9700 1 -- 172.21.15.38:0/2705449349 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318017d70 con 0x7fb3240abe50 2024-08-15T17:58:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fb320ff9700 1 -- 172.21.15.38:0/2705449349 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb318019040 con 0x7fb3240abe50 2024-08-15T17:58:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fb320ff9700 1 -- 172.21.15.38:0/2705449349 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31801f610 con 0x7fb3240abe50 2024-08-15T17:58:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fb32b25f700 1 -- 172.21.15.38:0/2705449349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3240af500 con 0x7fb3240abe50 2024-08-15T17:58:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fb32b25f700 1 -- 172.21.15.38:0/2705449349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3240afac0 con 0x7fb3240abe50 2024-08-15T17:58:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42fa43b700 1 --2- 172.21.15.38:0/3595831465 >> v2:172.21.15.104:3300/0 conn(0x7f42f4155720 0x7f42f40b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42f9439700 1 -- 172.21.15.38:0/3595831465 >> v2:172.21.15.104:3300/0 conn(0x7f42f4155720 msgr2=0x7f42f40b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42f9439700 1 --2- 172.21.15.38:0/3595831465 >> v2:172.21.15.104:3300/0 conn(0x7f42f4155720 0x7f42f40b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42f9439700 1 -- 172.21.15.38:0/3595831465 >> v1:172.21.15.38:6789/0 conn(0x7f42f406e9d0 legacy=0x7f42f4155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42f9439700 1 -- 172.21.15.38:0/3595831465 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f40ac310 con 0x7f42f40ab540 2024-08-15T17:58:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f42f9439700 1 --2- 172.21.15.38:0/3595831465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f40ab540 0x7f42f406eda0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f42f406b070 tx=0x7f42e800a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74218c183c64c7f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f7838989700 1 -- 172.21.15.38:0/761899318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782c0db430 con 0x7f782c0d7580 2024-08-15T17:58:41.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7f7e38ff9700 1 -- 172.21.15.38:0/2414091301 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7e2c01b3b0 con 0x7f7e3c0af190 2024-08-15T17:58:41.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fb313793700 1 -- 172.21.15.38:0/3313102658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb30c0645c0 con 0x7fb30c0b4220 2024-08-15T17:58:41.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc7bfff700 1 -- 172.21.15.38:0/580426609 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7400b640 con 0x7fdc840ab250 2024-08-15T17:58:41.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc7bfff700 1 -- 172.21.15.38:0/580426609 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7400b7a0 con 0x7fdc840ab250 2024-08-15T17:58:41.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc7bfff700 1 -- 172.21.15.38:0/580426609 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7400bb40 con 0x7fdc840ab250 2024-08-15T17:58:41.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc8a293700 1 -- 172.21.15.38:0/580426609 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc840ab250 msgr2=0x7fdc8406e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.674+0000 7fdc8a293700 1 --2- 172.21.15.38:0/580426609 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc840ab250 0x7fdc8406e850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc740098e0 tx=0x7fdc740091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fb3017fa700 1 -- 172.21.15.38:0/3313102658 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb2fc01b360 con 0x7fb30c0b4220 2024-08-15T17:58:41.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7f7e38ff9700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7e28060860 0x7f7e28062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7f7e38ff9700 1 -- 172.21.15.38:0/2414091301 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e2c07c320 con 0x7f7e3c0af190 2024-08-15T17:58:41.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fb3017fa700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb2f00608e0 0x7fb2f0062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 -- 172.21.15.38:0/580426609 shutdown_connections 2024-08-15T17:58:41.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 --2- 172.21.15.38:0/580426609 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc840ab250 0x7fdc8406e850 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 --2- 172.21.15.38:0/580426609 >> v2:172.21.15.104:3300/0 conn(0x7fdc840aae80 0x7fdc8406d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 -- 172.21.15.38:0/580426609 >> 172.21.15.38:0/580426609 conn(0x7fdc84066e00 msgr2=0x7fdc84067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7f7e40c48700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7e28060860 0x7f7e28062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fb3017fa700 1 -- 172.21.15.38:0/3313102658 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb2fc07c440 con 0x7fb30c0b4220 2024-08-15T17:58:41.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 -- 172.21.15.38:0/580426609 shutdown_connections 2024-08-15T17:58:41.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fb31152f700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb2f00608e0 0x7fb2f0062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 -- 172.21.15.38:0/580426609 wait complete. 2024-08-15T17:58:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7f7e40c48700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7e28060860 0x7f7e28062d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e34001390 tx=0x7f7e34002750 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fb31152f700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb2f00608e0 0x7fb2f0062d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3080097c0 tx=0x7fb308018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 Processor -- start 2024-08-15T17:58:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 -- start start 2024-08-15T17:58:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc840aae80 0x7fdc84139df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc840ab250 0x7fdc840ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc84161d50 0x7fdc840aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840abc70 con 0x7fdc840ab250 2024-08-15T17:58:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc841364b0 con 0x7fdc840aae80 2024-08-15T17:58:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc8a293700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc84097cc0 con 0x7fdc84161d50 2024-08-15T17:58:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc88a90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc840ab250 0x7fdc840ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc89a92700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc84161d50 0x7fdc840aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc88a90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc840ab250 0x7fdc840ab730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53834/0 (socket says 172.21.15.38:53834) 2024-08-15T17:58:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc88a90700 1 -- 172.21.15.38:0/318708592 learned_addr learned my addr 172.21.15.38:0/318708592 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc89291700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc840aae80 0x7fdc84139df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fb32b25f700 1 -- 172.21.15.38:0/2705449349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb324070360 con 0x7fb3240abe50 2024-08-15T17:58:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc88a90700 1 -- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc84161d50 msgr2=0x7fdc840aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc88a90700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc84161d50 0x7fdc840aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc88a90700 1 -- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc840aae80 msgr2=0x7fdc84139df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc88a90700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc840aae80 0x7fdc84139df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc88a90700 1 -- 172.21.15.38:0/318708592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc74009420 con 0x7fdc840ab250 2024-08-15T17:58:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fc6c8abe700 1 Processor -- start 2024-08-15T17:58:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c8abe700 1 -- start start 2024-08-15T17:58:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.678+0000 7fdc88a90700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc840ab250 0x7fdc840ab730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400a010 tx=0x7fdc74017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fb3017fa700 1 -- 172.21.15.38:0/3313102658 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb2fc04c8f0 con 0x7fb30c0b4220 2024-08-15T17:58:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c8abe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc6c4068840 0x7fc6c4068c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c8abe700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c40aa270 con 0x7fc6c40a9ce0 2024-08-15T17:58:41.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c8abe700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c40aa550 con 0x7fc6c40a9860 2024-08-15T17:58:41.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c8abe700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c40aa830 con 0x7fc6c4068840 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c3fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc6c40a9ce0 0x7fc6c40ab310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40170/0 (socket says 172.21.15.38:40170) 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c3fff700 1 -- 172.21.15.38:0/630628073 learned_addr learned my addr 172.21.15.38:0/630628073 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c37fe700 1 --2- 172.21.15.38:0/630628073 >> v2:172.21.15.104:3300/0 conn(0x7fc6c4068840 0x7fc6c4068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c37fe700 1 -- 172.21.15.38:0/630628073 >> v1:172.21.15.104:6789/0 conn(0x7fc6c40a9860 legacy=0x7fc6c4158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c37fe700 1 -- 172.21.15.38:0/630628073 >> v1:172.21.15.38:6789/0 conn(0x7fc6c40a9ce0 legacy=0x7fc6c40ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c37fe700 1 -- 172.21.15.38:0/630628073 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c40aab10 con 0x7fc6c4068840 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c37fe700 1 --2- 172.21.15.38:0/630628073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c4068840 0x7fc6c4068c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009910 tx=0x7fc6b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3c05959e356ae2e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c27fc700 1 -- 172.21.15.38:0/630628073 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b800b820 con 0x7fc6c4068840 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c27fc700 1 -- 172.21.15.38:0/630628073 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b800b980 con 0x7fc6c4068840 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c27fc700 1 -- 172.21.15.38:0/630628073 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b800bd20 con 0x7fc6c4068840 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c8abe700 1 -- 172.21.15.38:0/630628073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c4068840 msgr2=0x7fc6c4068c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.682+0000 7fc6c8abe700 1 --2- 172.21.15.38:0/630628073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c4068840 0x7fc6c4068c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009910 tx=0x7fc6b80091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a210fd700 1 Processor -- start 2024-08-15T17:58:41.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fc6c8abe700 1 -- 172.21.15.38:0/630628073 shutdown_connections 2024-08-15T17:58:41.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fc6c8abe700 1 --2- 172.21.15.38:0/630628073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c4068840 0x7fc6c4068c10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fc6c8abe700 1 -- 172.21.15.38:0/630628073 >> 172.21.15.38:0/630628073 conn(0x7fc6c406d800 msgr2=0x7fc6c406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fc6c8abe700 1 -- 172.21.15.38:0/630628073 shutdown_connections 2024-08-15T17:58:41.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fc6c8abe700 1 -- 172.21.15.38:0/630628073 wait complete. 2024-08-15T17:58:41.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f7e38ff9700 1 -- 172.21.15.38:0/2414091301 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7e2c04c7f0 con 0x7f7e3c0af190 2024-08-15T17:58:41.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fc6c8abe700 1 Processor -- start 2024-08-15T17:58:41.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fc6c8abe700 1 -- start start 2024-08-15T17:58:41.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fdc797fa700 1 -- 172.21.15.38:0/318708592 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7401bab0 con 0x7fdc840ab250 2024-08-15T17:58:41.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fdc797fa700 1 -- 172.21.15.38:0/318708592 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7401bc30 con 0x7fdc840ab250 2024-08-15T17:58:41.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fdc797fa700 1 -- 172.21.15.38:0/318708592 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc74025420 con 0x7fdc840ab250 2024-08-15T17:58:41.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fdc8a293700 1 -- 172.21.15.38:0/318708592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc840abea0 con 0x7fdc840ab250 2024-08-15T17:58:41.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fdc8a293700 1 -- 172.21.15.38:0/318708592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc840ac460 con 0x7fdc840ab250 2024-08-15T17:58:41.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7f42f8c38700 1 -- 172.21.15.38:0/3595831465 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8013a90 con 0x7f42f40ab540 2024-08-15T17:58:41.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7f42f8c38700 1 -- 172.21.15.38:0/3595831465 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e8013bf0 con 0x7f42f40ab540 2024-08-15T17:58:41.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7f42f8c38700 1 -- 172.21.15.38:0/3595831465 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e800c760 con 0x7f42f40ab540 2024-08-15T17:58:41.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f78227fc700 1 -- 172.21.15.38:0/761899318 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7828014aa0 con 0x7f782c0d7580 2024-08-15T17:58:41.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7f78227fc700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7814060930 0x7f7814062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7f78227fc700 1 -- 172.21.15.38:0/761899318 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7828018030 con 0x7f782c0d7580 2024-08-15T17:58:41.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7f783259c700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7814060930 0x7f7814062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a210fd700 1 -- start start 2024-08-15T17:58:41.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a210fd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c155cc0 0x7f1a1c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a210fd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1a1c068ce0 0x7f1a1c069a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a210fd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1a1c069f90 0x7f1a1c0c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a210fd700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1c06a400 con 0x7f1a1c068ce0 2024-08-15T17:58:41.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a210fd700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1c069180 con 0x7f1a1c069f90 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a210fd700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c069460 con 0x7f1a1c155cc0 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a1b59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1a1c069f90 0x7f1a1c0c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a1b59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1a1c069f90 0x7f1a1c0c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:36004/0 (socket says 172.21.15.38:36004) 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a1b59e700 1 -- 172.21.15.38:0/1764801709 learned_addr learned my addr 172.21.15.38:0/1764801709 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a1a59c700 1 --2- 172.21.15.38:0/1764801709 >> v2:172.21.15.38:3300/0 conn(0x7f1a1c068ce0 0x7f1a1c069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a1b59e700 1 -- 172.21.15.38:0/1764801709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c155cc0 msgr2=0x7f1a1c0687a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a1b59e700 1 --2- 172.21.15.38:0/1764801709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c155cc0 0x7f1a1c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a1b59e700 1 -- 172.21.15.38:0/1764801709 >> v2:172.21.15.38:3300/0 conn(0x7f1a1c068ce0 msgr2=0x7f1a1c069a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a1b59e700 1 --2- 172.21.15.38:0/1764801709 >> v2:172.21.15.38:3300/0 conn(0x7f1a1c068ce0 0x7f1a1c069a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a1b59e700 1 -- 172.21.15.38:0/1764801709 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c069740 con 0x7f1a1c069f90 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a1a59c700 1 --2- 172.21.15.38:0/1764801709 >> v2:172.21.15.38:3300/0 conn(0x7f1a1c068ce0 0x7f1a1c069a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7f1a1b59e700 1 --2- 172.21.15.38:0/1764801709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a1c069f90 0x7f1a1c0c7dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0b31f0 tx=0x7f1a14009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28bd7ec53ea62108 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c8abe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c4068840 0x7fc6c416c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c8abe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c40a9860 0x7fc6c40c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c8abe700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6c40a9ce0 0x7fc6c406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c8abe700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c416ca40 con 0x7fc6c40a9860 2024-08-15T17:58:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c8abe700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40c5aa0 con 0x7fc6c4068840 2024-08-15T17:58:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c8abe700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40c5d80 con 0x7fc6c40a9ce0 2024-08-15T17:58:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c4068840 0x7fc6c416c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c4068840 0x7fc6c416c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:36018/0 (socket says 172.21.15.38:36018) 2024-08-15T17:58:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c3fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6c40a9ce0 0x7fc6c406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c37fe700 1 -- 172.21.15.38:0/3283057140 learned_addr learned my addr 172.21.15.38:0/3283057140 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fb320ff9700 1 -- 172.21.15.38:0/2705449349 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb31801f980 con 0x7fb3240abe50 2024-08-15T17:58:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fb320ff9700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb310060860 0x7fb310062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fb320ff9700 1 -- 172.21.15.38:0/2705449349 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb3180853a0 con 0x7fb3240abe50 2024-08-15T17:58:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.686+0000 7fb323fff700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb310060860 0x7fb310062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fb323fff700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb310060860 0x7fb310062d10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb324179630 tx=0x7fb31c0018a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c37fe700 1 -- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6c40a9ce0 msgr2=0x7fc6c406a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c37fe700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6c40a9ce0 0x7fc6c406a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c37fe700 1 -- 172.21.15.38:0/3283057140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c40a9860 msgr2=0x7fc6c40c51c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T17:58:41.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c37fe700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c40a9860 0x7fc6c40c51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c37fe700 1 -- 172.21.15.38:0/3283057140 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b8009420 con 0x7fc6c4068840 2024-08-15T17:58:41.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7f783259c700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7814060930 0x7f7814062de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7824005130 tx=0x7f7824016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.690+0000 7fc6c37fe700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c4068840 0x7fc6c416c500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4069650 tx=0x7fc6b8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.694+0000 7f78227fc700 1 -- 172.21.15.38:0/761899318 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f782804e130 con 0x7f782c0d7580 2024-08-15T17:58:41.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.694+0000 7fdc8a293700 1 -- 172.21.15.38:0/318708592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc640052b0 con 0x7fdc840ab250 2024-08-15T17:58:41.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a19d9b700 1 -- 172.21.15.38:0/1764801709 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1400e070 con 0x7f1a1c069f90 2024-08-15T17:58:41.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a19d9b700 1 -- 172.21.15.38:0/1764801709 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1400ba50 con 0x7f1a1c069f90 2024-08-15T17:58:41.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a19d9b700 1 -- 172.21.15.38:0/1764801709 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a140092e0 con 0x7f1a1c069f90 2024-08-15T17:58:41.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fb320ff9700 1 -- 172.21.15.38:0/2705449349 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb3180557f0 con 0x7fb3240abe50 2024-08-15T17:58:41.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fc6b37fe700 1 -- 172.21.15.38:0/3283057140 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8024440 con 0x7fc6c4068840 2024-08-15T17:58:41.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fc6b37fe700 1 -- 172.21.15.38:0/3283057140 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b80245a0 con 0x7fc6c4068840 2024-08-15T17:58:41.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fc6b37fe700 1 -- 172.21.15.38:0/3283057140 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8024960 con 0x7fc6c4068840 2024-08-15T17:58:41.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fc6c8abe700 1 -- 172.21.15.38:0/3283057140 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c40c6060 con 0x7fc6c4068840 2024-08-15T17:58:41.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fc6c8abe700 1 -- 172.21.15.38:0/3283057140 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c40c66b0 con 0x7fc6c4068840 2024-08-15T17:58:41.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f42fac3c700 1 -- 172.21.15.38:0/3595831465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f40ab540 msgr2=0x7f42f406eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f42fac3c700 1 --2- 172.21.15.38:0/3595831465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f40ab540 0x7f42f406eda0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f42f406b070 tx=0x7f42e800a3f0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a210fd700 1 -- 172.21.15.38:0/1764801709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a1c069f90 msgr2=0x7f1a1c0c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a210fd700 1 --2- 172.21.15.38:0/1764801709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a1c069f90 0x7f1a1c0c7dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0b31f0 tx=0x7f1a14009840 comp rx=0 tx=0).stop 2024-08-15T17:58:41.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a210fd700 1 -- 172.21.15.38:0/1764801709 shutdown_connections 2024-08-15T17:58:41.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a210fd700 1 --2- 172.21.15.38:0/1764801709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a1c069f90 0x7f1a1c0c7dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a210fd700 1 --2- 172.21.15.38:0/1764801709 >> v2:172.21.15.38:3300/0 conn(0x7f1a1c068ce0 0x7f1a1c069a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a210fd700 1 --2- 172.21.15.38:0/1764801709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c155cc0 0x7f1a1c0687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a210fd700 1 -- 172.21.15.38:0/1764801709 >> 172.21.15.38:0/1764801709 conn(0x7f1a1c0cda30 msgr2=0x7f1a1c0c9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fdc797fa700 1 -- 172.21.15.38:0/318708592 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdc74023070 con 0x7fdc840ab250 2024-08-15T17:58:41.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fc6c8abe700 1 -- 172.21.15.38:0/3283057140 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c40638f0 con 0x7fc6c4068840 2024-08-15T17:58:41.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f42fac3c700 1 -- 172.21.15.38:0/3595831465 shutdown_connections 2024-08-15T17:58:41.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f42fac3c700 1 --2- 172.21.15.38:0/3595831465 >> v2:172.21.15.104:3300/0 conn(0x7f42f4155720 0x7f42f40b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f42fac3c700 1 --2- 172.21.15.38:0/3595831465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f40ab540 0x7f42f406eda0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f42fac3c700 1 -- 172.21.15.38:0/3595831465 >> 172.21.15.38:0/3595831465 conn(0x7f42f40678d0 msgr2=0x7f42f4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f42fac3c700 1 -- 172.21.15.38:0/3595831465 shutdown_connections 2024-08-15T17:58:41.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a210fd700 1 -- 172.21.15.38:0/1764801709 shutdown_connections 2024-08-15T17:58:41.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f1a210fd700 1 -- 172.21.15.38:0/1764801709 wait complete. 2024-08-15T17:58:41.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fdc797fa700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc68060860 0x7fdc68062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fdc797fa700 1 -- 172.21.15.38:0/318708592 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdc7407e2e0 con 0x7fdc840ab250 2024-08-15T17:58:41.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fdc89291700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc68060860 0x7fdc68062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fc6b37fe700 1 -- 172.21.15.38:0/3283057140 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc6b801a420 con 0x7fc6c4068840 2024-08-15T17:58:41.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7f42fac3c700 1 -- 172.21.15.38:0/3595831465 wait complete. 2024-08-15T17:58:41.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.698+0000 7fdc89291700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc68060860 0x7fdc68062d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00f570 tx=0x7fdc7c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7fc6b37fe700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6a8060960 0x7fc6a8062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7fc6b37fe700 1 -- 172.21.15.38:0/3283057140 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc6b807f060 con 0x7fc6c4068840 2024-08-15T17:58:41.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a210fd700 1 Processor -- start 2024-08-15T17:58:41.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a210fd700 1 -- start start 2024-08-15T17:58:41.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fac3c700 1 Processor -- start 2024-08-15T17:58:41.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fac3c700 1 -- start start 2024-08-15T17:58:41.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7fc6c2ffd700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6a8060960 0x7fc6a8062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fac3c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42f406e9d0 0x7f42f4165120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fac3c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42f40ab540 0x7f42f41573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fac3c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f4155720 0x7f42f4159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fac3c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f4166050 con 0x7f42f4155720 2024-08-15T17:58:41.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fac3c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f41657b0 con 0x7f42f40ab540 2024-08-15T17:58:41.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fac3c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f4165a90 con 0x7f42f406e9d0 2024-08-15T17:58:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42f9c3a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42f406e9d0 0x7f42f4165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42f9c3a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42f406e9d0 0x7f42f4165120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:44174/0 (socket says 172.21.15.38:44174) 2024-08-15T17:58:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42f9c3a700 1 -- 172.21.15.38:0/1031073234 learned_addr learned my addr 172.21.15.38:0/1031073234 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42f9439700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42f40ab540 0x7f42f41573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fa43b700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f4155720 0x7f42f4159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fa43b700 1 -- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42f406e9d0 msgr2=0x7f42f4165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fa43b700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42f406e9d0 0x7f42f4165120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fa43b700 1 -- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42f40ab540 msgr2=0x7f42f41573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fa43b700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42f40ab540 0x7f42f41573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f42fa43b700 1 -- 172.21.15.38:0/1031073234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e800a690 con 0x7f42f4155720 2024-08-15T17:58:41.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a210fd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a1c068ce0 0x7f1a1c148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a210fd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c069f90 0x7f1a1c14d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a210fd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a1c155cc0 0x7f1a1c16a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a210fd700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c14f430 con 0x7f1a1c068ce0 2024-08-15T17:58:41.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a1ad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a1c068ce0 0x7f1a1c148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a210fd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c14cbe0 con 0x7f1a1c155cc0 2024-08-15T17:58:41.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a210fd700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0b2070 con 0x7f1a1c069f90 2024-08-15T17:58:41.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a1ad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a1c068ce0 0x7f1a1c148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53860/0 (socket says 172.21.15.38:53860) 2024-08-15T17:58:41.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a1ad9d700 1 -- 172.21.15.38:0/3018628299 learned_addr learned my addr 172.21.15.38:0/3018628299 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:41.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a1a59c700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c069f90 0x7f1a1c14d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a1b59e700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a1c155cc0 0x7f1a1c16a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a1ad9d700 1 -- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c069f90 msgr2=0x7f1a1c14d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a1ad9d700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c069f90 0x7f1a1c14d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a1ad9d700 1 -- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a1c155cc0 msgr2=0x7f1a1c16a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a1ad9d700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a1c155cc0 0x7f1a1c16a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a1ad9d700 1 -- 172.21.15.38:0/3018628299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a14009ae0 con 0x7f1a1c068ce0 2024-08-15T17:58:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.702+0000 7f1a1ad9d700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a1c068ce0 0x7f1a1c148950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c011e40 tx=0x7f1a0c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.706+0000 7f42fa43b700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f4155720 0x7f42f4159310 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00ecc0 tx=0x7f42ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.706+0000 7fc6c2ffd700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6a8060960 0x7fc6a8062e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0013d0 tx=0x7fc6bc007e60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.706+0000 7f42e5ffb700 1 -- 172.21.15.38:0/1031073234 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec00c8c0 con 0x7f42f4155720 2024-08-15T17:58:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.706+0000 7f42e5ffb700 1 -- 172.21.15.38:0/1031073234 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec00f040 con 0x7f42f4155720 2024-08-15T17:58:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.706+0000 7fdc797fa700 1 -- 172.21.15.38:0/318708592 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdc7404e7b0 con 0x7fdc840ab250 2024-08-15T17:58:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.706+0000 7f42e5ffb700 1 -- 172.21.15.38:0/1031073234 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec00cc90 con 0x7f42f4155720 2024-08-15T17:58:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.706+0000 7f42fac3c700 1 -- 172.21.15.38:0/1031073234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f4165d70 con 0x7f42f4155720 2024-08-15T17:58:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.706+0000 7f42fac3c700 1 -- 172.21.15.38:0/1031073234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f4159850 con 0x7f42f4155720 2024-08-15T17:58:41.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.710+0000 7fc6b37fe700 1 -- 172.21.15.38:0/3283057140 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc6b804f610 con 0x7fc6c4068840 2024-08-15T17:58:41.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.710+0000 7f42e5ffb700 1 -- 172.21.15.38:0/1031073234 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f42ec014690 con 0x7f42f4155720 2024-08-15T17:58:41.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.714+0000 7f1a02ffd700 1 -- 172.21.15.38:0/3018628299 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c007a40 con 0x7f1a1c068ce0 2024-08-15T17:58:41.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.714+0000 7f1a02ffd700 1 -- 172.21.15.38:0/3018628299 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c007ba0 con 0x7f1a1c068ce0 2024-08-15T17:58:41.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.714+0000 7f1a210fd700 1 -- 172.21.15.38:0/3018628299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1c16af10 con 0x7f1a1c068ce0 2024-08-15T17:58:41.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.714+0000 7f1a210fd700 1 -- 172.21.15.38:0/3018628299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1c14c870 con 0x7f1a1c068ce0 2024-08-15T17:58:41.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.714+0000 7f1a02ffd700 1 -- 172.21.15.38:0/3018628299 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c0110d0 con 0x7f1a1c068ce0 2024-08-15T17:58:41.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.710+0000 7f42e5ffb700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f42d80608e0 0x7f42d8062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.710+0000 7f42e5ffb700 1 -- 172.21.15.38:0/1031073234 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f42ec07d630 con 0x7f42f4155720 2024-08-15T17:58:41.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.710+0000 7f42f9c3a700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f42d80608e0 0x7f42d8062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.710+0000 7f42f9c3a700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f42d80608e0 0x7f42d8062d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000c00 tx=0x7f42f0012870 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.722+0000 7f42d77fe700 1 -- 172.21.15.38:0/1031073234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f4166550 con 0x7f42f4155720 2024-08-15T17:58:41.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.722+0000 7f1a210fd700 1 -- 172.21.15.38:0/3018628299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19fc0052b0 con 0x7f1a1c068ce0 2024-08-15T17:58:41.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.722+0000 7f1a02ffd700 1 -- 172.21.15.38:0/3018628299 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1a0c011480 con 0x7f1a1c068ce0 2024-08-15T17:58:41.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.722+0000 7f1a02ffd700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1a08060960 0x7f1a08062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:41.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.722+0000 7f1a02ffd700 1 -- 172.21.15.38:0/3018628299 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a0c07cb10 con 0x7f1a1c068ce0 2024-08-15T17:58:41.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.722+0000 7f1a1a59c700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1a08060960 0x7f1a08062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:41.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.726+0000 7f1a1a59c700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1a08060960 0x7f1a08062e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0b2f20 tx=0x7f1a10008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:41.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.730+0000 7f42e5ffb700 1 -- 172.21.15.38:0/1031073234 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f42ec04d9e0 con 0x7f42f4155720 2024-08-15T17:58:41.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.738+0000 7f1a02ffd700 1 -- 172.21.15.38:0/3018628299 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1a0c04cfc0 con 0x7f1a1c068ce0 2024-08-15T17:58:41.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.982+0000 7fb313793700 1 -- 172.21.15.38:0/3313102658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb30c0aa830 con 0x7fb30c0b4220 2024-08-15T17:58:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.982+0000 7fb3017fa700 1 -- 172.21.15.38:0/3313102658 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb2fc018090 con 0x7fb30c0b4220 2024-08-15T17:58:41.989 INFO:teuthology.orchestra.run.smithi038.stdout:60129542149 2024-08-15T17:58:41.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 -- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb2f00608e0 msgr2=0x7fb2f0062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb2f00608e0 0x7fb2f0062d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3080097c0 tx=0x7fb308018000 comp rx=0 tx=0).stop 2024-08-15T17:58:41.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 -- 172.21.15.38:0/3313102658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb30c0b4220 msgr2=0x7fb30c0af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:41.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb30c0b4220 0x7fb30c0af540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00c7e0 tx=0x7fb2fc001a00 comp rx=0 tx=0).stop 2024-08-15T17:58:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 -- 172.21.15.38:0/3313102658 shutdown_connections 2024-08-15T17:58:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb2f00608e0 0x7fb2f0062d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb30c158f00 0x7fb30c0afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb30c0b4220 0x7fb30c0af540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 --2- 172.21.15.38:0/3313102658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb30c0b0aa0 0x7fb30c0ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 -- 172.21.15.38:0/3313102658 >> 172.21.15.38:0/3313102658 conn(0x7fb30c099510 msgr2=0x7fb30c157ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 -- 172.21.15.38:0/3313102658 shutdown_connections 2024-08-15T17:58:41.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:41.990+0000 7fb2f6ffd700 1 -- 172.21.15.38:0/3313102658 wait complete. 2024-08-15T17:58:42.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2024-08-15T17:58:42.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.010+0000 7f7e41c4a700 1 -- 172.21.15.38:0/2414091301 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7e3c0643a0 con 0x7f7e3c0af190 2024-08-15T17:58:42.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.010+0000 7f7e38ff9700 1 -- 172.21.15.38:0/2414091301 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7e2c017090 con 0x7f7e3c0af190 2024-08-15T17:58:42.012 INFO:teuthology.orchestra.run.smithi038.stdout:60129542150 2024-08-15T17:58:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.010+0000 7f7838989700 1 -- 172.21.15.38:0/761899318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f782c005800 con 0x7f782c0d7580 2024-08-15T17:58:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f78227fc700 1 -- 172.21.15.38:0/761899318 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7828022090 con 0x7f782c0d7580 2024-08-15T17:58:42.014 INFO:teuthology.orchestra.run.smithi038.stdout:60129542150 2024-08-15T17:58:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 -- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7e28060860 msgr2=0x7f7e28062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7e28060860 0x7f7e28062d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e34001390 tx=0x7f7e34002750 comp rx=0 tx=0).stop 2024-08-15T17:58:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 -- 172.21.15.38:0/2414091301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e3c0af190 msgr2=0x7f7e3c0869b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e3c0af190 0x7f7e3c0869b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00c560 tx=0x7f7e2c007d50 comp rx=0 tx=0).stop 2024-08-15T17:58:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 -- 172.21.15.38:0/2414091301 shutdown_connections 2024-08-15T17:58:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7e28060860 0x7f7e28062d10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e3c0af190 0x7f7e3c0869b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e3c06e570 0x7f7e3c14ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 --2- 172.21.15.38:0/2414091301 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7e3c06e1a0 0x7f7e3c14dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 -- 172.21.15.38:0/2414091301 >> 172.21.15.38:0/2414091301 conn(0x7f7e3c0678d0 msgr2=0x7f7e3c0692b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 -- 172.21.15.38:0/2414091301 shutdown_connections 2024-08-15T17:58:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7e41c4a700 1 -- 172.21.15.38:0/2414091301 wait complete. 2024-08-15T17:58:42.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 -- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7814060930 msgr2=0x7f7814062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7814060930 0x7f7814062de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7824005130 tx=0x7f7824016000 comp rx=0 tx=0).stop 2024-08-15T17:58:42.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 -- 172.21.15.38:0/761899318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782c0d7580 msgr2=0x7f782c0dd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782c0d7580 0x7f782c0dd550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f782c00dc20 tx=0x7f78280091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 -- 172.21.15.38:0/761899318 shutdown_connections 2024-08-15T17:58:42.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7814060930 0x7f7814062de0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f782c0d7580 0x7f782c0dd550 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f782c0d7110 0x7f782c0dd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 --2- 172.21.15.38:0/761899318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f781c003bf0 0x7f782c0d69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 -- 172.21.15.38:0/761899318 >> 172.21.15.38:0/761899318 conn(0x7f782c038be0 msgr2=0x7f782c054f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:42.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 -- 172.21.15.38:0/761899318 shutdown_connections 2024-08-15T17:58:42.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.014+0000 7f7838989700 1 -- 172.21.15.38:0/761899318 wait complete. 2024-08-15T17:58:42.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.022+0000 7fdc8a293700 1 -- 172.21.15.38:0/318708592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdc64005830 con 0x7fdc840ab250 2024-08-15T17:58:42.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.022+0000 7fdc797fa700 1 -- 172.21.15.38:0/318708592 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdc74051dd0 con 0x7fdc840ab250 2024-08-15T17:58:42.024 INFO:teuthology.orchestra.run.smithi038.stdout:60129542149 2024-08-15T17:58:42.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 -- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc68060860 msgr2=0x7fdc68062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc68060860 0x7fdc68062d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00f570 tx=0x7fdc7c00b040 comp rx=0 tx=0).stop 2024-08-15T17:58:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 -- 172.21.15.38:0/318708592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc840ab250 msgr2=0x7fdc840ab730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc840ab250 0x7fdc840ab730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400a010 tx=0x7fdc74017000 comp rx=0 tx=0).stop 2024-08-15T17:58:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 -- 172.21.15.38:0/318708592 shutdown_connections 2024-08-15T17:58:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc68060860 0x7fdc68062d10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc84161d50 0x7fdc840aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc840ab250 0x7fdc840ab730 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 --2- 172.21.15.38:0/318708592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc840aae80 0x7fdc84139df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 -- 172.21.15.38:0/318708592 >> 172.21.15.38:0/318708592 conn(0x7fdc84066e00 msgr2=0x7fdc8415f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:42.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 -- 172.21.15.38:0/318708592 shutdown_connections 2024-08-15T17:58:42.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fdc8a293700 1 -- 172.21.15.38:0/318708592 wait complete. 2024-08-15T17:58:42.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.5 2024-08-15T17:58:42.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fb32b25f700 1 -- 172.21.15.38:0/2705449349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb3240645c0 con 0x7fb3240abe50 2024-08-15T17:58:42.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.0 2024-08-15T17:58:42.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.026+0000 7fb320ff9700 1 -- 172.21.15.38:0/2705449349 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb318058e10 con 0x7fb3240abe50 2024-08-15T17:58:42.031 INFO:teuthology.orchestra.run.smithi038.stdout:64424509446 2024-08-15T17:58:42.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 -- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb310060860 msgr2=0x7fb310062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb310060860 0x7fb310062d10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb324179630 tx=0x7fb31c0018a0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 -- 172.21.15.38:0/2705449349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3240abe50 msgr2=0x7fb3240aa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3240abe50 0x7fb3240aa0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb318017180 tx=0x7fb318008040 comp rx=0 tx=0).stop 2024-08-15T17:58:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 -- 172.21.15.38:0/2705449349 shutdown_connections 2024-08-15T17:58:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb310060860 0x7fb310062d10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb324166980 0x7fb3241591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb3240ac220 0x7fb3240aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 --2- 172.21.15.38:0/2705449349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3240abe50 0x7fb3240aa0d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 -- 172.21.15.38:0/2705449349 >> 172.21.15.38:0/2705449349 conn(0x7fb324068300 msgr2=0x7fb324166270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 -- 172.21.15.38:0/2705449349 shutdown_connections 2024-08-15T17:58:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.030+0000 7fb32b25f700 1 -- 172.21.15.38:0/2705449349 wait complete. 2024-08-15T17:58:42.042 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.2 2024-08-15T17:58:42.042 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-08-15T17:58:42.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.042+0000 7fc6c8abe700 1 -- 172.21.15.38:0/3283057140 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc6c406e590 con 0x7fc6c4068840 2024-08-15T17:58:42.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.042+0000 7fc6b37fe700 1 -- 172.21.15.38:0/3283057140 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc6b8015090 con 0x7fc6c4068840 2024-08-15T17:58:42.044 INFO:teuthology.orchestra.run.smithi038.stdout:64424509447 2024-08-15T17:58:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 -- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6a8060960 msgr2=0x7fc6a8062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6a8060960 0x7fc6a8062e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0013d0 tx=0x7fc6bc007e60 comp rx=0 tx=0).stop 2024-08-15T17:58:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 -- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c4068840 msgr2=0x7fc6c416c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c4068840 0x7fc6c416c500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4069650 tx=0x7fc6b8017000 comp rx=0 tx=0).stop 2024-08-15T17:58:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 -- 172.21.15.38:0/3283057140 shutdown_connections 2024-08-15T17:58:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc6a8060960 0x7fc6a8062e10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6c40a9ce0 0x7fc6c406a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c40a9860 0x7fc6c40c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 --2- 172.21.15.38:0/3283057140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c4068840 0x7fc6c416c500 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 -- 172.21.15.38:0/3283057140 >> 172.21.15.38:0/3283057140 conn(0x7fc6c406d800 msgr2=0x7fc6c41409e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 -- 172.21.15.38:0/3283057140 shutdown_connections 2024-08-15T17:58:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.046+0000 7fc6c8abe700 1 -- 172.21.15.38:0/3283057140 wait complete. 2024-08-15T17:58:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 -- 172.21.15.38:0/1031073234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f42f40643a0 con 0x7f42f4155720 2024-08-15T17:58:42.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42e5ffb700 1 -- 172.21.15.38:0/1031073234 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f42ec018030 con 0x7f42f4155720 2024-08-15T17:58:42.051 INFO:teuthology.orchestra.run.smithi038.stdout:60129542151 2024-08-15T17:58:42.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 -- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f42d80608e0 msgr2=0x7f42d8062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f42d80608e0 0x7f42d8062d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000c00 tx=0x7f42f0012870 comp rx=0 tx=0).stop 2024-08-15T17:58:42.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 -- 172.21.15.38:0/1031073234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f4155720 msgr2=0x7f42f4159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f4155720 0x7f42f4159310 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00ecc0 tx=0x7f42ec00b040 comp rx=0 tx=0).stop 2024-08-15T17:58:42.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 -- 172.21.15.38:0/1031073234 shutdown_connections 2024-08-15T17:58:42.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f42d80608e0 0x7f42d8062d90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f4155720 0x7f42f4159310 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42f40ab540 0x7f42f41573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 --2- 172.21.15.38:0/1031073234 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42f406e9d0 0x7f42f4165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 -- 172.21.15.38:0/1031073234 >> 172.21.15.38:0/1031073234 conn(0x7f42f40678d0 msgr2=0x7f42f40adf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:42.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 -- 172.21.15.38:0/1031073234 shutdown_connections 2024-08-15T17:58:42.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.050+0000 7f42d77fe700 1 -- 172.21.15.38:0/1031073234 wait complete. 2024-08-15T17:58:42.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.7 2024-08-15T17:58:42.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.058+0000 7f1a210fd700 1 -- 172.21.15.38:0/3018628299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f19fc005830 con 0x7f1a1c068ce0 2024-08-15T17:58:42.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.058+0000 7f1a02ffd700 1 -- 172.21.15.38:0/3018628299 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a0c0505e0 con 0x7f1a1c068ce0 2024-08-15T17:58:42.062 INFO:teuthology.orchestra.run.smithi038.stdout:64424509446 2024-08-15T17:58:42.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.3 2024-08-15T17:58:42.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 -- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1a08060960 msgr2=0x7f1a08062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1a08060960 0x7f1a08062e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0b2f20 tx=0x7f1a10008040 comp rx=0 tx=0).stop 2024-08-15T17:58:42.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 -- 172.21.15.38:0/3018628299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a1c068ce0 msgr2=0x7f1a1c148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:42.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a1c068ce0 0x7f1a1c148950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c011e40 tx=0x7f1a0c011af0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 -- 172.21.15.38:0/3018628299 shutdown_connections 2024-08-15T17:58:42.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1a08060960 0x7f1a08062e10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a1c155cc0 0x7f1a1c16a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c069f90 0x7f1a1c14d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 --2- 172.21.15.38:0/3018628299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a1c068ce0 0x7f1a1c148950 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 -- 172.21.15.38:0/3018628299 >> 172.21.15.38:0/3018628299 conn(0x7f1a1c0cda30 msgr2=0x7f1a1c0683e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 -- 172.21.15.38:0/3018628299 shutdown_connections 2024-08-15T17:58:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:42.062+0000 7f1a210fd700 1 -- 172.21.15.38:0/3018628299 wait complete. 2024-08-15T17:58:42.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-08-15T17:58:43.007 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-15T17:58:43.030 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-15T17:58:43.031 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-15T17:58:43.043 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-15T17:58:43.043 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-15T17:58:43.060 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-15T17:58:43.065 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-15T17:58:43.074 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-15T17:58:43.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.134+0000 7f6aee490700 1 Processor -- start 2024-08-15T17:58:43.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.134+0000 7f6aee490700 1 -- start start 2024-08-15T17:58:43.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6aee490700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ae813b630 0x7f6ae80b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6aee490700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6ae813bf10 0x7f6ae806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6aee490700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6ae80b1950 0x7f6ae806e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6aee490700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae813cf90 con 0x7f6ae813bf10 2024-08-15T17:58:43.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6aee490700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae813c380 con 0x7f6ae80b1950 2024-08-15T17:58:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6aee490700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae813c660 con 0x7f6ae813b630 2024-08-15T17:58:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6ae77fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6ae813bf10 0x7f6ae806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6ae77fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6ae813bf10 0x7f6ae806df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53874/0 (socket says 172.21.15.38:53874) 2024-08-15T17:58:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6ae77fe700 1 -- 172.21.15.38:0/3341566385 learned_addr learned my addr 172.21.15.38:0/3341566385 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6ae77fe700 1 -- 172.21.15.38:0/3341566385 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ae813b630 msgr2=0x7f6ae80b1410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-15T17:58:43.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6ae77fe700 1 --2- 172.21.15.38:0/3341566385 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ae813b630 0x7f6ae80b1410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6ae77fe700 1 -- 172.21.15.38:0/3341566385 >> v2:172.21.15.104:3300/0 conn(0x7f6ae80b1950 msgr2=0x7f6ae806e450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-15T17:58:43.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6ae77fe700 1 --2- 172.21.15.38:0/3341566385 >> v2:172.21.15.104:3300/0 conn(0x7f6ae80b1950 0x7f6ae806e450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6ae77fe700 1 -- 172.21.15.38:0/3341566385 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae813c940 con 0x7f6ae813bf10 2024-08-15T17:58:43.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6ae77fe700 1 --2- 172.21.15.38:0/3341566385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ae813bf10 0x7f6ae806df10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ae806ed70 tx=0x7f6ae0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc1d02e9b910c79 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6ae6ffd700 1 -- 172.21.15.38:0/3341566385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00092e0 con 0x7f6ae813bf10 2024-08-15T17:58:43.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.138+0000 7f6ae6ffd700 1 -- 172.21.15.38:0/3341566385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae000ca80 con 0x7f6ae813bf10 2024-08-15T17:58:43.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 -- 172.21.15.38:0/3341566385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ae813bf10 msgr2=0x7f6ae806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 --2- 172.21.15.38:0/3341566385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ae813bf10 0x7f6ae806df10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ae806ed70 tx=0x7f6ae0009f90 comp rx=0 tx=0).stop 2024-08-15T17:58:43.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 -- 172.21.15.38:0/3341566385 shutdown_connections 2024-08-15T17:58:43.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 --2- 172.21.15.38:0/3341566385 >> v2:172.21.15.104:3300/0 conn(0x7f6ae80b1950 0x7f6ae806e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 --2- 172.21.15.38:0/3341566385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ae813bf10 0x7f6ae806df10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 --2- 172.21.15.38:0/3341566385 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ae813b630 0x7f6ae80b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 -- 172.21.15.38:0/3341566385 >> 172.21.15.38:0/3341566385 conn(0x7f6ae80678d0 msgr2=0x7f6ae8067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 -- 172.21.15.38:0/3341566385 shutdown_connections 2024-08-15T17:58:43.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 -- 172.21.15.38:0/3341566385 wait complete. 2024-08-15T17:58:43.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 Processor -- start 2024-08-15T17:58:43.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 -- start start 2024-08-15T17:58:43.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.142+0000 7f6aee490700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae80b1950 0x7f6ae81565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6aee490700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ae813b630 0x7f6ae813a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6aee490700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ae813bf10 0x7f6ae81653a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ae7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae80b1950 0x7f6ae81565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ae7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae80b1950 0x7f6ae81565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:36040/0 (socket says 172.21.15.38:36040) 2024-08-15T17:58:43.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ae7fff700 1 -- 172.21.15.38:0/2518165544 learned_addr learned my addr 172.21.15.38:0/2518165544 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ae77fe700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ae813b630 0x7f6ae813a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6aee490700 1 -- 172.21.15.38:0/2518165544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae8157510 con 0x7f6ae813bf10 2024-08-15T17:58:43.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6aee490700 1 -- 172.21.15.38:0/2518165544 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae80c7e10 con 0x7f6ae80b1950 2024-08-15T17:58:43.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6aee490700 1 -- 172.21.15.38:0/2518165544 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae8158a10 con 0x7f6ae813b630 2024-08-15T17:58:43.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6aeca2d700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ae813bf10 0x7f6ae81653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ae77fe700 1 -- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae80b1950 msgr2=0x7f6ae81565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ae77fe700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae80b1950 0x7f6ae81565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ae77fe700 1 -- 172.21.15.38:0/2518165544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ae813bf10 msgr2=0x7f6ae81653a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ae77fe700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ae813bf10 0x7f6ae81653a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ae77fe700 1 -- 172.21.15.38:0/2518165544 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae000c370 con 0x7f6ae813b630 2024-08-15T17:58:43.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6aeca2d700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ae813bf10 0x7f6ae81653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:43.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ae7fff700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae80b1950 0x7f6ae81565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:43.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ae77fe700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ae813b630 0x7f6ae813a590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0017000 tx=0x7f6ae001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6ad7fff700 1 -- 172.21.15.38:0/2518165544 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae001c040 con 0x7f6ae813b630 2024-08-15T17:58:43.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6aee490700 1 -- 172.21.15.38:0/2518165544 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae8156c70 con 0x7f6ae813b630 2024-08-15T17:58:43.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.146+0000 7f6aee490700 1 -- 172.21.15.38:0/2518165544 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae8157230 con 0x7f6ae813b630 2024-08-15T17:58:43.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.150+0000 7f6ad7fff700 1 -- 172.21.15.38:0/2518165544 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae000f650 con 0x7f6ae813b630 2024-08-15T17:58:43.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.150+0000 7f6ad7fff700 1 -- 172.21.15.38:0/2518165544 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000ccc0 con 0x7f6ae813b630 2024-08-15T17:58:43.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.150+0000 7f6ad7fff700 1 -- 172.21.15.38:0/2518165544 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6ae000fc60 con 0x7f6ae813b630 2024-08-15T17:58:43.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.150+0000 7f6ad7fff700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6acc0609b0 0x7f6acc062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.150+0000 7f6ad7fff700 1 -- 172.21.15.38:0/2518165544 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6ae0013070 con 0x7f6ae813b630 2024-08-15T17:58:43.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.150+0000 7f6ae7fff700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6acc0609b0 0x7f6acc062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.150+0000 7f6ad5ffb700 1 -- 172.21.15.38:0/2518165544 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac40052b0 con 0x7f6ae813b630 2024-08-15T17:58:43.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.154+0000 7f6ae7fff700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6acc0609b0 0x7f6acc062e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009f60 tx=0x7f6ad80091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.166+0000 7f6ad7fff700 1 -- 172.21.15.38:0/2518165544 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6ae004f400 con 0x7f6ae813b630 2024-08-15T17:58:43.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96f106c700 1 Processor -- start 2024-08-15T17:58:43.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96f106c700 1 -- start start 2024-08-15T17:58:43.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96f106c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96ec158fb0 0x7f96ec0aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96f106c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f96ec15d910 0x7f96ec06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96f106c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec0ad380 con 0x7f96ec06e450 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96f106c700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec06e8f0 con 0x7f96ec15d910 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96f106c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0ab2d0 con 0x7f96ec158fb0 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96eb7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f96ec15d910 0x7f96ec06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96eb7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f96ec15d910 0x7f96ec06df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53890/0 (socket says 172.21.15.38:53890) 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96eb7fe700 1 -- 172.21.15.38:0/17964329 learned_addr learned my addr 172.21.15.38:0/17964329 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96ebfff700 1 --2- 172.21.15.38:0/17964329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96ec158fb0 0x7f96ec0aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96f086b700 1 --1- 172.21.15.38:0/17964329 >> v1:172.21.15.38:6789/0 conn(0x7f96ec06e450 0x7f96ec0a9860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40174/0 (socket says 172.21.15.38:40174) 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96eb7fe700 1 -- 172.21.15.38:0/17964329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96ec158fb0 msgr2=0x7f96ec0aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96eb7fe700 1 --2- 172.21.15.38:0/17964329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96ec158fb0 0x7f96ec0aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96eb7fe700 1 -- 172.21.15.38:0/17964329 >> v1:172.21.15.38:6789/0 conn(0x7f96ec06e450 legacy=0x7f96ec0a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96eb7fe700 1 -- 172.21.15.38:0/17964329 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec0ab5b0 con 0x7f96ec15d910 2024-08-15T17:58:43.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96eb7fe700 1 --2- 172.21.15.38:0/17964329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ec15d910 0x7f96ec06df10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009f20 tx=0x7f96e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa51db6d1f2bd13 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96eaffd700 1 -- 172.21.15.38:0/17964329 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e400e070 con 0x7f96ec15d910 2024-08-15T17:58:43.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96eaffd700 1 -- 172.21.15.38:0/17964329 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e400b9d0 con 0x7f96ec15d910 2024-08-15T17:58:43.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.178+0000 7f96eaffd700 1 -- 172.21.15.38:0/17964329 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e40092e0 con 0x7f96ec15d910 2024-08-15T17:58:43.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 -- 172.21.15.38:0/17964329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ec15d910 msgr2=0x7f96ec06df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 --2- 172.21.15.38:0/17964329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ec15d910 0x7f96ec06df10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009f20 tx=0x7f96e4009840 comp rx=0 tx=0).stop 2024-08-15T17:58:43.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 -- 172.21.15.38:0/17964329 shutdown_connections 2024-08-15T17:58:43.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 --2- 172.21.15.38:0/17964329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ec15d910 0x7f96ec06df10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 --2- 172.21.15.38:0/17964329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96ec158fb0 0x7f96ec0aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 -- 172.21.15.38:0/17964329 >> 172.21.15.38:0/17964329 conn(0x7f96ec0678d0 msgr2=0x7f96ec067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 -- 172.21.15.38:0/17964329 shutdown_connections 2024-08-15T17:58:43.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 -- 172.21.15.38:0/17964329 wait complete. 2024-08-15T17:58:43.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 Processor -- start 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 -- start start 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96ec06e450 0x7f96ec179bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ec158fb0 0x7f96ec176c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96ec15d910 0x7f96ec1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0cb4e0 con 0x7f96ec158fb0 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0cb200 con 0x7f96ec15d910 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.182+0000 7f96f106c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0c9130 con 0x7f96ec06e450 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96eb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ec158fb0 0x7f96ec176c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96eb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ec158fb0 0x7f96ec176c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53894/0 (socket says 172.21.15.38:53894) 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96eb7fe700 1 -- 172.21.15.38:0/2342586702 learned_addr learned my addr 172.21.15.38:0/2342586702 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96f086b700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96ec15d910 0x7f96ec1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96ebfff700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96ec06e450 0x7f96ec179bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96eb7fe700 1 -- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96ec06e450 msgr2=0x7f96ec179bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96eb7fe700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96ec06e450 0x7f96ec179bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96eb7fe700 1 -- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96ec15d910 msgr2=0x7f96ec1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96eb7fe700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96ec15d910 0x7f96ec1568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96eb7fe700 1 -- 172.21.15.38:0/2342586702 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e4009ae0 con 0x7f96ec158fb0 2024-08-15T17:58:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96ebfff700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96ec06e450 0x7f96ec179bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96eb7fe700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ec158fb0 0x7f96ec176c30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96ec06beb0 tx=0x7f96e4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96e3fff700 1 -- 172.21.15.38:0/2342586702 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e40088e0 con 0x7f96ec158fb0 2024-08-15T17:58:43.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96e3fff700 1 -- 172.21.15.38:0/2342586702 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e4008a60 con 0x7f96ec158fb0 2024-08-15T17:58:43.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96e3fff700 1 -- 172.21.15.38:0/2342586702 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4008e20 con 0x7f96ec158fb0 2024-08-15T17:58:43.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96f106c700 1 -- 172.21.15.38:0/2342586702 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec0c9410 con 0x7f96ec158fb0 2024-08-15T17:58:43.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7f96f106c700 1 -- 172.21.15.38:0/2342586702 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec0c96f0 con 0x7f96ec158fb0 2024-08-15T17:58:43.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7ff4b1e4c700 1 Processor -- start 2024-08-15T17:58:43.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.186+0000 7ff4b1e4c700 1 -- start start 2024-08-15T17:58:43.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f96f106c700 1 -- 172.21.15.38:0/2342586702 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec0643a0 con 0x7f96ec158fb0 2024-08-15T17:58:43.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f96e3fff700 1 -- 172.21.15.38:0/2342586702 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f96e4003040 con 0x7f96ec158fb0 2024-08-15T17:58:43.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f96e3fff700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f96cc060930 0x7f96cc062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f96e3fff700 1 -- 172.21.15.38:0/2342586702 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f96e407ead0 con 0x7f96ec158fb0 2024-08-15T17:58:43.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f411b59e700 1 Processor -- start 2024-08-15T17:58:43.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f411b59e700 1 -- start start 2024-08-15T17:58:43.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f96ebfff700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f96cc060930 0x7f96cc062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4b1e4c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff4ac06e620 0x7ff4ac164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4b1e4c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ac0ad380 con 0x7ff4ac158fb0 2024-08-15T17:58:43.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4b1e4c700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ac0ab2d0 con 0x7ff4ac15d910 2024-08-15T17:58:43.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4b1e4c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ac0ab5b0 con 0x7ff4ac06e620 2024-08-15T17:58:43.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4abfff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff4ac15d910 0x7ff4ac06df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:59658/0 (socket says 172.21.15.38:59658) 2024-08-15T17:58:43.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4abfff700 1 -- 172.21.15.38:0/2821020399 learned_addr learned my addr 172.21.15.38:0/2821020399 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:43.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4b164b700 1 --2- 172.21.15.38:0/2821020399 >> v2:172.21.15.104:3300/0 conn(0x7ff4ac06e620 0x7ff4ac164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f96ebfff700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f96cc060930 0x7f96cc062de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f96d4011480 tx=0x7f96d40091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4b164b700 1 -- 172.21.15.38:0/2821020399 >> v1:172.21.15.104:6789/0 conn(0x7ff4ac15d910 legacy=0x7ff4ac06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4b164b700 1 -- 172.21.15.38:0/2821020399 >> v1:172.21.15.38:6789/0 conn(0x7ff4ac158fb0 legacy=0x7ff4ac0aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:43.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4b164b700 1 -- 172.21.15.38:0/2821020399 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac0ab890 con 0x7ff4ac06e620 2024-08-15T17:58:43.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f411b59e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f411c13e6b0 0x7f411c0b1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4b164b700 1 --2- 172.21.15.38:0/2821020399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4ac06e620 0x7ff4ac164000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0098e0 tx=0x7ff49c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1af67011e6c0cbd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f411b59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f411c13ea80 0x7f411c156b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f411b59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c156700 con 0x7f411c155670 2024-08-15T17:58:43.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f411b59e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c155af0 con 0x7f411c13e6b0 2024-08-15T17:58:43.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f411b59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c155dd0 con 0x7f411c13ea80 2024-08-15T17:58:43.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f411ad9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f411c13ea80 0x7f411c156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f411ad9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f411c13ea80 0x7f411c156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:36064/0 (socket says 172.21.15.38:36064) 2024-08-15T17:58:43.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f411ad9d700 1 -- 172.21.15.38:0/3335058879 learned_addr learned my addr 172.21.15.38:0/3335058879 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4ab7fe700 1 -- 172.21.15.38:0/2821020399 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c009e80 con 0x7ff4ac06e620 2024-08-15T17:58:43.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4ab7fe700 1 -- 172.21.15.38:0/2821020399 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c00b820 con 0x7ff4ac06e620 2024-08-15T17:58:43.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7ff4ab7fe700 1 -- 172.21.15.38:0/2821020399 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00bba0 con 0x7ff4ac06e620 2024-08-15T17:58:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.190+0000 7f411a59c700 1 --2- 172.21.15.38:0/3335058879 >> v2:172.21.15.38:3300/0 conn(0x7f411c13e6b0 0x7f411c0b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411ad9d700 1 -- 172.21.15.38:0/3335058879 >> v2:172.21.15.38:3300/0 conn(0x7f411c13e6b0 msgr2=0x7f411c0b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411ad9d700 1 --2- 172.21.15.38:0/3335058879 >> v2:172.21.15.38:3300/0 conn(0x7f411c13e6b0 0x7f411c0b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411ad9d700 1 -- 172.21.15.38:0/3335058879 >> v1:172.21.15.104:6789/0 conn(0x7f411c155670 legacy=0x7f411c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411ad9d700 1 -- 172.21.15.38:0/3335058879 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c1560b0 con 0x7f411c13ea80 2024-08-15T17:58:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411a59c700 1 --2- 172.21.15.38:0/3335058879 >> v2:172.21.15.38:3300/0 conn(0x7f411c13e6b0 0x7f411c0b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411ad9d700 1 --2- 172.21.15.38:0/3335058879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f411c13ea80 0x7f411c156b90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f411c156670 tx=0x7f4114007820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=330bfe78e4b031a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/2821020399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4ac06e620 msgr2=0x7ff4ac164000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7ff4b1e4c700 1 --2- 172.21.15.38:0/2821020399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4ac06e620 0x7ff4ac164000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0098e0 tx=0x7ff49c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/2821020399 shutdown_connections 2024-08-15T17:58:43.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7ff4b1e4c700 1 --2- 172.21.15.38:0/2821020399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4ac06e620 0x7ff4ac164000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/2821020399 >> 172.21.15.38:0/2821020399 conn(0x7ff4ac0678d0 msgr2=0x7ff4ac067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411959a700 1 -- 172.21.15.38:0/3335058879 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411400a140 con 0x7f411c13ea80 2024-08-15T17:58:43.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411959a700 1 -- 172.21.15.38:0/3335058879 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411400a2a0 con 0x7f411c13ea80 2024-08-15T17:58:43.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411959a700 1 -- 172.21.15.38:0/3335058879 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411400a640 con 0x7f411c13ea80 2024-08-15T17:58:43.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/2821020399 shutdown_connections 2024-08-15T17:58:43.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/2821020399 wait complete. 2024-08-15T17:58:43.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411b59e700 1 -- 172.21.15.38:0/3335058879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f411c13ea80 msgr2=0x7f411c156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411b59e700 1 --2- 172.21.15.38:0/3335058879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f411c13ea80 0x7f411c156b90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f411c156670 tx=0x7f4114007820 comp rx=0 tx=0).stop 2024-08-15T17:58:43.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411b59e700 1 -- 172.21.15.38:0/3335058879 shutdown_connections 2024-08-15T17:58:43.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411b59e700 1 --2- 172.21.15.38:0/3335058879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f411c13ea80 0x7f411c156b90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411b59e700 1 --2- 172.21.15.38:0/3335058879 >> v2:172.21.15.38:3300/0 conn(0x7f411c13e6b0 0x7f411c0b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411b59e700 1 -- 172.21.15.38:0/3335058879 >> 172.21.15.38:0/3335058879 conn(0x7f411c0678d0 msgr2=0x7f411c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7ff4b1e4c700 1 Processor -- start 2024-08-15T17:58:43.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411b59e700 1 -- 172.21.15.38:0/3335058879 shutdown_connections 2024-08-15T17:58:43.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7ff4b1e4c700 1 -- start start 2024-08-15T17:58:43.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.194+0000 7f411b59e700 1 -- 172.21.15.38:0/3335058879 wait complete. 2024-08-15T17:58:43.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b1e4c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4ac06e620 0x7ff4ac16cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f96e3fff700 1 -- 172.21.15.38:0/2342586702 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f96e404efa0 con 0x7f96ec158fb0 2024-08-15T17:58:43.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b0e4a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4ac06e620 0x7ff4ac16cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b0e4a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4ac06e620 0x7ff4ac16cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53920/0 (socket says 172.21.15.38:53920) 2024-08-15T17:58:43.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b0e4a700 1 -- 172.21.15.38:0/415533037 learned_addr learned my addr 172.21.15.38:0/415533037 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b1e4c700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4ac158fb0 0x7ff4ac1477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f411b59e700 1 Processor -- start 2024-08-15T17:58:43.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f411b59e700 1 -- start start 2024-08-15T17:58:43.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b1e4c700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4ac15d910 0x7ff4ac0a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac0c8840 con 0x7ff4ac06e620 2024-08-15T17:58:43.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac16da10 con 0x7ff4ac15d910 2024-08-15T17:58:43.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac147d10 con 0x7ff4ac158fb0 2024-08-15T17:58:43.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4abfff700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4ac158fb0 0x7ff4ac1477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b164b700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4ac15d910 0x7ff4ac0a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b0e4a700 1 -- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4ac158fb0 msgr2=0x7ff4ac1477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b0e4a700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4ac158fb0 0x7ff4ac1477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b0e4a700 1 -- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4ac15d910 msgr2=0x7ff4ac0a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b0e4a700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4ac15d910 0x7ff4ac0a9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b0e4a700 1 -- 172.21.15.38:0/415533037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c009420 con 0x7ff4ac06e620 2024-08-15T17:58:43.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b164b700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4ac15d910 0x7ff4ac0a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:43.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b0e4a700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4ac06e620 0x7ff4ac16cae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400bbb0 tx=0x7ff4a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4a8ff9700 1 -- 172.21.15.38:0/415533037 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40092e0 con 0x7ff4ac06e620 2024-08-15T17:58:43.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4a8ff9700 1 -- 172.21.15.38:0/415533037 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a400c7a0 con 0x7ff4ac06e620 2024-08-15T17:58:43.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4a8ff9700 1 -- 172.21.15.38:0/415533037 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400cb40 con 0x7ff4ac06e620 2024-08-15T17:58:43.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ac16d170 con 0x7ff4ac06e620 2024-08-15T17:58:43.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ac16d730 con 0x7ff4ac06e620 2024-08-15T17:58:43.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ac0643a0 con 0x7ff4ac06e620 2024-08-15T17:58:43.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f411b59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f411c13e6b0 0x7f411c1363f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f411b59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f411c13ea80 0x7f411c154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f411b59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f411c0c0de0 0x7f411c0c1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f4119d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f411c13ea80 0x7f411c154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f4119d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f411c13ea80 0x7f411c154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53926/0 (socket says 172.21.15.38:53926) 2024-08-15T17:58:43.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f4119d9b700 1 -- 172.21.15.38:0/4024675295 learned_addr learned my addr 172.21.15.38:0/4024675295 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1397f0 con 0x7f411c13ea80 2024-08-15T17:58:43.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c136930 con 0x7f411c13e6b0 2024-08-15T17:58:43.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1659e0 con 0x7f411c0c0de0 2024-08-15T17:58:43.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7ff4a8ff9700 1 -- 172.21.15.38:0/415533037 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff4a4012070 con 0x7ff4ac06e620 2024-08-15T17:58:43.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7ff4a8ff9700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff498060860 0x7ff498062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f4119d9b700 1 -- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f411c0c0de0 msgr2=0x7f411c0c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f4119d9b700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f411c0c0de0 0x7f411c0c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f4119d9b700 1 -- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f411c13e6b0 msgr2=0x7f411c1363f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-15T17:58:43.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f4119d9b700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f411c13e6b0 0x7f411c1363f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f4119d9b700 1 -- 172.21.15.38:0/4024675295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c00b6d0 con 0x7f411c13ea80 2024-08-15T17:58:43.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f4119d9b700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f411c13ea80 0x7f411c154400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f411000b5e0 tx=0x7f41100090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7ff4a8ff9700 1 -- 172.21.15.38:0/415533037 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff4a407c9b0 con 0x7ff4ac06e620 2024-08-15T17:58:43.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7ff4abfff700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff498060860 0x7ff498062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7ff4abfff700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff498060860 0x7ff498062d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac068a80 tx=0x7ff4a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.206+0000 7ff4a8ff9700 1 -- 172.21.15.38:0/415533037 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff4a404cd60 con 0x7ff4ac06e620 2024-08-15T17:58:43.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f410a7fc700 1 -- 172.21.15.38:0/4024675295 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411000f510 con 0x7f411c13ea80 2024-08-15T17:58:43.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4114007ac0 con 0x7f411c13ea80 2024-08-15T17:58:43.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c165fa0 con 0x7f411c13ea80 2024-08-15T17:58:43.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f8c01f3e700 1 Processor -- start 2024-08-15T17:58:43.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f410a7fc700 1 -- 172.21.15.38:0/4024675295 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411000f690 con 0x7f411c13ea80 2024-08-15T17:58:43.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.198+0000 7f8c01f3e700 1 -- start start 2024-08-15T17:58:43.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f8c01f3e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8bfc0af190 0x7f8bfc165700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f8c01f3e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc166dc0 con 0x7f8bfc06e570 2024-08-15T17:58:43.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f8c01f3e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc166490 con 0x7f8bfc06e1a0 2024-08-15T17:58:43.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f8c01f3e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bfc166770 con 0x7f8bfc0af190 2024-08-15T17:58:43.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f8c0173d700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f8bfc06e570 0x7f8bfc165c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40184/0 (socket says 172.21.15.38:40184) 2024-08-15T17:58:43.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f8c0173d700 1 -- 172.21.15.38:0/1566099415 learned_addr learned my addr 172.21.15.38:0/1566099415 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:43.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.202+0000 7f8bfbfff700 1 --2- 172.21.15.38:0/1566099415 >> v2:172.21.15.104:3300/0 conn(0x7f8bfc0af190 0x7f8bfc165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f8bfbfff700 1 -- 172.21.15.38:0/1566099415 >> v1:172.21.15.104:6789/0 conn(0x7f8bfc06e1a0 legacy=0x7f8bfc0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f8bfbfff700 1 -- 172.21.15.38:0/1566099415 >> v1:172.21.15.38:6789/0 conn(0x7f8bfc06e570 legacy=0x7f8bfc165c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f8bfbfff700 1 -- 172.21.15.38:0/1566099415 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc166a50 con 0x7f8bfc0af190 2024-08-15T17:58:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.206+0000 7f8bfb7fe700 1 -- 172.21.15.38:0/1566099415 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199063788 0 0) 0x7f8bfc166dc0 con 0x7f8bfc06e570 2024-08-15T17:58:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f8bfbfff700 1 --2- 172.21.15.38:0/1566099415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8bfc0af190 0x7f8bfc165700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40098c0 tx=0x7f8bf4009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ca25b673bb575fa server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f8bfb7fe700 1 -- 172.21.15.38:0/1566099415 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4011600 con 0x7f8bfc0af190 2024-08-15T17:58:43.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f410a7fc700 1 -- 172.21.15.38:0/4024675295 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411000fa50 con 0x7f411c13ea80 2024-08-15T17:58:43.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f410a7fc700 1 -- 172.21.15.38:0/4024675295 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f411001a400 con 0x7f411c13ea80 2024-08-15T17:58:43.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f8bfb7fe700 1 -- 172.21.15.38:0/1566099415 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4011760 con 0x7f8bfc0af190 2024-08-15T17:58:43.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f410a7fc700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f40fc060a60 0x7f40fc062f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c0643a0 con 0x7f411c13ea80 2024-08-15T17:58:43.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1566099415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8bfc0af190 msgr2=0x7f8bfc165700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.210+0000 7f8c01f3e700 1 --2- 172.21.15.38:0/1566099415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8bfc0af190 0x7f8bfc165700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40098c0 tx=0x7f8bf4009bd0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.214+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1566099415 shutdown_connections 2024-08-15T17:58:43.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.214+0000 7f8c01f3e700 1 --2- 172.21.15.38:0/1566099415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8bfc0af190 0x7f8bfc165700 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.214+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1566099415 >> 172.21.15.38:0/1566099415 conn(0x7f8bfc0678d0 msgr2=0x7f8bfc067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.214+0000 7f410a7fc700 1 -- 172.21.15.38:0/4024675295 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f411007c760 con 0x7f411c13ea80 2024-08-15T17:58:43.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.214+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1566099415 shutdown_connections 2024-08-15T17:58:43.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.214+0000 7f411a59c700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f40fc060a60 0x7f40fc062f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.214+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1566099415 wait complete. 2024-08-15T17:58:43.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.214+0000 7f411a59c700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f40fc060a60 0x7f40fc062f10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f411c068640 tx=0x7f410c00b430 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.214+0000 7f8c01f3e700 1 Processor -- start 2024-08-15T17:58:43.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.214+0000 7f8c01f3e700 1 -- start start 2024-08-15T17:58:43.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8c01f3e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8bfc06e1a0 0x7f8bfc086a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8c01f3e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bfc06e570 0x7f8bfc161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8c01f3e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8bfc0af190 0x7f8bfc161b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8c01f3e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc153430 con 0x7f8bfc06e570 2024-08-15T17:58:43.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8bfbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bfc06e570 0x7f8bfc161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8bfbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bfc06e570 0x7f8bfc161610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53940/0 (socket says 172.21.15.38:53940) 2024-08-15T17:58:43.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8bfbfff700 1 -- 172.21.15.38:0/1035202108 learned_addr learned my addr 172.21.15.38:0/1035202108 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc152820 con 0x7f8bfc0af190 2024-08-15T17:58:43.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8c0173d700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8bfc0af190 0x7f8bfc161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8c00f3c700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8bfc06e1a0 0x7f8bfc086a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc152b00 con 0x7f8bfc06e1a0 2024-08-15T17:58:43.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8bfbfff700 1 -- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8bfc06e1a0 msgr2=0x7f8bfc086a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8bfbfff700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8bfc06e1a0 0x7f8bfc086a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8bfbfff700 1 -- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8bfc0af190 msgr2=0x7f8bfc161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8bfbfff700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8bfc0af190 0x7f8bfc161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8bfbfff700 1 -- 172.21.15.38:0/1035202108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf4009e70 con 0x7f8bfc06e570 2024-08-15T17:58:43.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f8bfbfff700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bfc06e570 0x7f8bfc161610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc06beb0 tx=0x7f8bf4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.218+0000 7f410a7fc700 1 -- 172.21.15.38:0/4024675295 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f41100506a0 con 0x7f411c13ea80 2024-08-15T17:58:43.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.222+0000 7f8bf8ff9700 1 -- 172.21.15.38:0/1035202108 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4011600 con 0x7f8bfc06e570 2024-08-15T17:58:43.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.222+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc152de0 con 0x7f8bfc06e570 2024-08-15T17:58:43.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.222+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc086f50 con 0x7f8bfc06e570 2024-08-15T17:58:43.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.222+0000 7f8bf8ff9700 1 -- 172.21.15.38:0/1035202108 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4020070 con 0x7f8bfc06e570 2024-08-15T17:58:43.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.222+0000 7f8bf8ff9700 1 -- 172.21.15.38:0/1035202108 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf401b6a0 con 0x7f8bfc06e570 2024-08-15T17:58:43.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.222+0000 7f8bf8ff9700 1 -- 172.21.15.38:0/1035202108 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8bf4022030 con 0x7f8bfc06e570 2024-08-15T17:58:43.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.222+0000 7f8bf8ff9700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8be8060930 0x7f8be8062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.222+0000 7f8bf8ff9700 1 -- 172.21.15.38:0/1035202108 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8bf400e070 con 0x7f8bfc06e570 2024-08-15T17:58:43.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.226+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc0052b0 con 0x7f8bfc06e570 2024-08-15T17:58:43.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.226+0000 7f8c00f3c700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8be8060930 0x7f8be8062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.226+0000 7f8c00f3c700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8be8060930 0x7f8be8062de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0003b50 tx=0x7f8bf0007e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7f8bf8ff9700 1 -- 172.21.15.38:0/1035202108 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8bf404f510 con 0x7f8bfc06e570 2024-08-15T17:58:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7fda2138b700 1 Processor -- start 2024-08-15T17:58:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7f14d6f72700 1 Processor -- start 2024-08-15T17:58:43.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7f14d6f72700 1 -- start start 2024-08-15T17:58:43.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7f14d6f72700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f14d01593b0 0x7f14d015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7f14d6f72700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14d015a8b0 0x7f14d00aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7f14d6f72700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d00acae0 con 0x7f14d00aa980 2024-08-15T17:58:43.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7f14d6f72700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d00aade0 con 0x7f14d01593b0 2024-08-15T17:58:43.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7f14d6f72700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00ac650 con 0x7f14d015a8b0 2024-08-15T17:58:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7fda2138b700 1 -- start start 2024-08-15T17:58:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7fda2138b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fda1c13ea80 0x7fda1c160040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7fda2138b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1c156700 con 0x7fda1c13e6b0 2024-08-15T17:58:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7fda2138b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1c155af0 con 0x7fda1c155670 2024-08-15T17:58:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7fda2138b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1c155dd0 con 0x7fda1c13ea80 2024-08-15T17:58:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7fda1b7fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fda1c155670 0x7fda1c06df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:59678/0 (socket says 172.21.15.38:59678) 2024-08-15T17:58:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7fda1b7fe700 1 -- 172.21.15.38:0/2365041687 learned_addr learned my addr 172.21.15.38:0/2365041687 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7f14d550f700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f14d00aa980 0x7f14d00b0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40198/0 (socket says 172.21.15.38:40198) 2024-08-15T17:58:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7f14d550f700 1 -- 172.21.15.38:0/3027208080 learned_addr learned my addr 172.21.15.38:0/3027208080 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7fda20b8a700 1 --2- 172.21.15.38:0/2365041687 >> v2:172.21.15.104:3300/0 conn(0x7fda1c13ea80 0x7fda1c160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.234+0000 7f14cffff700 1 --2- 172.21.15.38:0/3027208080 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14d015a8b0 0x7f14d00aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14d4d0e700 1 --2- 172.21.15.38:0/3027208080 >> v2:172.21.15.38:3300/0 conn(0x7f14d01593b0 0x7f14d015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14cf7fe700 1 -- 172.21.15.38:0/3027208080 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477138834 0 0) 0x7f14d00acae0 con 0x7f14d00aa980 2024-08-15T17:58:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14cf7fe700 1 -- 172.21.15.38:0/3027208080 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14b8003570 con 0x7f14d00aa980 2024-08-15T17:58:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14d4d0e700 1 -- 172.21.15.38:0/3027208080 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14d015a8b0 msgr2=0x7f14d00aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14d4d0e700 1 --2- 172.21.15.38:0/3027208080 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14d015a8b0 0x7f14d00aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14d4d0e700 1 -- 172.21.15.38:0/3027208080 >> v1:172.21.15.38:6789/0 conn(0x7f14d00aa980 legacy=0x7f14d00b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14d4d0e700 1 -- 172.21.15.38:0/3027208080 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d0159780 con 0x7f14d01593b0 2024-08-15T17:58:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14cffff700 1 --2- 172.21.15.38:0/3027208080 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14d015a8b0 0x7f14d00aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:43.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14d4d0e700 1 --2- 172.21.15.38:0/3027208080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d01593b0 0x7f14d015a370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f14c00098e0 tx=0x7f14c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce87f58945f5ba96 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7fda20b8a700 1 -- 172.21.15.38:0/2365041687 >> v1:172.21.15.104:6789/0 conn(0x7fda1c155670 legacy=0x7fda1c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7fda20b8a700 1 -- 172.21.15.38:0/2365041687 >> v1:172.21.15.38:6789/0 conn(0x7fda1c13e6b0 legacy=0x7fda1c0b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7fda20b8a700 1 -- 172.21.15.38:0/2365041687 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c1560b0 con 0x7fda1c13ea80 2024-08-15T17:58:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14cf7fe700 1 -- 172.21.15.38:0/3027208080 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0011630 con 0x7f14d01593b0 2024-08-15T17:58:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14cf7fe700 1 -- 172.21.15.38:0/3027208080 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c0011790 con 0x7f14d01593b0 2024-08-15T17:58:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14cf7fe700 1 -- 172.21.15.38:0/3027208080 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0011b50 con 0x7f14d01593b0 2024-08-15T17:58:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14d6f72700 1 -- 172.21.15.38:0/3027208080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d01593b0 msgr2=0x7f14d015a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7f14d6f72700 1 --2- 172.21.15.38:0/3027208080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d01593b0 0x7f14d015a370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f14c00098e0 tx=0x7f14c00091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.238+0000 7fda20b8a700 1 --2- 172.21.15.38:0/2365041687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda1c13ea80 0x7fda1c160040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fda100098e0 tx=0x7fda100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff14bc8d5897e2c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.242+0000 7fda1affd700 1 -- 172.21.15.38:0/2365041687 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10009e80 con 0x7fda1c13ea80 2024-08-15T17:58:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.242+0000 7fda1affd700 1 -- 172.21.15.38:0/2365041687 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1000b760 con 0x7fda1c13ea80 2024-08-15T17:58:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.242+0000 7fda2138b700 1 -- 172.21.15.38:0/2365041687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda1c13ea80 msgr2=0x7fda1c160040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.242+0000 7fda2138b700 1 --2- 172.21.15.38:0/2365041687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda1c13ea80 0x7fda1c160040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fda100098e0 tx=0x7fda100091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.242+0000 7fda2138b700 1 -- 172.21.15.38:0/2365041687 shutdown_connections 2024-08-15T17:58:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.242+0000 7fda2138b700 1 --2- 172.21.15.38:0/2365041687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda1c13ea80 0x7fda1c160040 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.242+0000 7fda2138b700 1 -- 172.21.15.38:0/2365041687 >> 172.21.15.38:0/2365041687 conn(0x7fda1c0678d0 msgr2=0x7fda1c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.242+0000 7fda2138b700 1 -- 172.21.15.38:0/2365041687 shutdown_connections 2024-08-15T17:58:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.242+0000 7fda2138b700 1 -- 172.21.15.38:0/2365041687 wait complete. 2024-08-15T17:58:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.242+0000 7fda2138b700 1 Processor -- start 2024-08-15T17:58:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.242+0000 7fda2138b700 1 -- start start 2024-08-15T17:58:43.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda2138b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda1c13e6b0 0x7fda1c152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda2138b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda1c13ea80 0x7fda1c12ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7f14d6f72700 1 -- 172.21.15.38:0/3027208080 shutdown_connections 2024-08-15T17:58:43.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7f14d6f72700 1 --2- 172.21.15.38:0/3027208080 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14d015a8b0 0x7f14d00aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7f14d6f72700 1 --2- 172.21.15.38:0/3027208080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d01593b0 0x7f14d015a370 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7f14d6f72700 1 -- 172.21.15.38:0/3027208080 >> 172.21.15.38:0/3027208080 conn(0x7f14d0068300 msgr2=0x7f14d00a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda2138b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda1c12bfe0 0x7fda1c0ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda2138b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c152ca0 con 0x7fda1c12bfe0 2024-08-15T17:58:43.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda2138b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c1519a0 con 0x7fda1c13ea80 2024-08-15T17:58:43.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda2138b700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c151c80 con 0x7fda1c13e6b0 2024-08-15T17:58:43.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda1b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda1c13ea80 0x7fda1c12ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda1bfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda1c13e6b0 0x7fda1c152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda1bfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda1c13e6b0 0x7fda1c152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:44248/0 (socket says 172.21.15.38:44248) 2024-08-15T17:58:43.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda1bfff700 1 -- 172.21.15.38:0/3197313007 learned_addr learned my addr 172.21.15.38:0/3197313007 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda20b8a700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda1c12bfe0 0x7fda1c0ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7f14d6f72700 1 -- 172.21.15.38:0/3027208080 shutdown_connections 2024-08-15T17:58:43.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda1b7fe700 1 -- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda1c13e6b0 msgr2=0x7fda1c152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda1b7fe700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda1c13e6b0 0x7fda1c152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda1b7fe700 1 -- 172.21.15.38:0/3197313007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda1c12bfe0 msgr2=0x7fda1c0ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda1b7fe700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda1c12bfe0 0x7fda1c0ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda1b7fe700 1 -- 172.21.15.38:0/3197313007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda10009420 con 0x7fda1c13ea80 2024-08-15T17:58:43.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7fda1b7fe700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda1c13ea80 0x7fda1c12ed00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fda14001c80 tx=0x7fda140061b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7f14d6f72700 1 -- 172.21.15.38:0/3027208080 wait complete. 2024-08-15T17:58:43.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7f14d6f72700 1 Processor -- start 2024-08-15T17:58:43.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7fda03fff700 1 -- 172.21.15.38:0/3197313007 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140080d0 con 0x7fda1c13ea80 2024-08-15T17:58:43.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.246+0000 7f14d6f72700 1 -- start start 2024-08-15T17:58:43.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d6f72700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d00aa980 0x7f14d00b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d6f72700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14d01593b0 0x7f14d00afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d6f72700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14d015a8b0 0x7f14d0156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d6f72700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00b00e0 con 0x7f14d00aa980 2024-08-15T17:58:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7fda03fff700 1 -- 172.21.15.38:0/3197313007 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda14008250 con 0x7fda1c13ea80 2024-08-15T17:58:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7fda03fff700 1 -- 172.21.15.38:0/3197313007 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1400d8a0 con 0x7fda1c13ea80 2024-08-15T17:58:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7fda2138b700 1 -- 172.21.15.38:0/3197313007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1c151f60 con 0x7fda1c13ea80 2024-08-15T17:58:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7fda2138b700 1 -- 172.21.15.38:0/3197313007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1c12f240 con 0x7fda1c13ea80 2024-08-15T17:58:43.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d6f72700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00b1760 con 0x7f14d015a8b0 2024-08-15T17:58:43.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d6f72700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00b0b50 con 0x7f14d01593b0 2024-08-15T17:58:43.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d550f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14d015a8b0 0x7f14d0156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d550f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14d015a8b0 0x7f14d0156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:36120/0 (socket says 172.21.15.38:36120) 2024-08-15T17:58:43.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d550f700 1 -- 172.21.15.38:0/656546076 learned_addr learned my addr 172.21.15.38:0/656546076 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14cffff700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14d01593b0 0x7f14d00afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d4d0e700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d00aa980 0x7f14d00b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d4d0e700 1 -- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14d01593b0 msgr2=0x7f14d00afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d4d0e700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14d01593b0 0x7f14d00afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d4d0e700 1 -- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14d015a8b0 msgr2=0x7f14d0156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d4d0e700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14d015a8b0 0x7f14d0156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d4d0e700 1 -- 172.21.15.38:0/656546076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c4009420 con 0x7f14d00aa980 2024-08-15T17:58:43.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d4d0e700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d00aa980 0x7f14d00b0450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14c0016000 tx=0x7f14c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14ccff9700 1 -- 172.21.15.38:0/656546076 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0011900 con 0x7f14d00aa980 2024-08-15T17:58:43.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14ccff9700 1 -- 172.21.15.38:0/656546076 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c0011a80 con 0x7f14d00aa980 2024-08-15T17:58:43.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d6f72700 1 -- 172.21.15.38:0/656546076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c0009420 con 0x7f14d00aa980 2024-08-15T17:58:43.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14ccff9700 1 -- 172.21.15.38:0/656546076 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0026680 con 0x7f14d00aa980 2024-08-15T17:58:43.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7f14d6f72700 1 -- 172.21.15.38:0/656546076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d00b1110 con 0x7f14d00aa980 2024-08-15T17:58:43.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.250+0000 7fda03fff700 1 -- 172.21.15.38:0/3197313007 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fda1400dc50 con 0x7fda1c13ea80 2024-08-15T17:58:43.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.254+0000 7fda03fff700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fda080609b0 0x7fda08062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.254+0000 7fda03fff700 1 -- 172.21.15.38:0/3197313007 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fda1407c460 con 0x7fda1c13ea80 2024-08-15T17:58:43.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.254+0000 7fda1bfff700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fda080609b0 0x7fda08062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.254+0000 7f14ccff9700 1 -- 172.21.15.38:0/656546076 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f14c0023030 con 0x7f14d00aa980 2024-08-15T17:58:43.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.254+0000 7fda1bfff700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fda080609b0 0x7fda08062e60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fda1c13d9a0 tx=0x7fda0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.254+0000 7f14ccff9700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f14b4060860 0x7f14b4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.254+0000 7f14ccff9700 1 -- 172.21.15.38:0/656546076 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f14c007e160 con 0x7f14d00aa980 2024-08-15T17:58:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.254+0000 7f14cffff700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f14b4060860 0x7f14b4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.254+0000 7f14cffff700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f14b4060860 0x7f14b4062d10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14d0099a90 tx=0x7f14c40091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.258+0000 7f14be7fc700 1 -- 172.21.15.38:0/656546076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d00b1910 con 0x7f14d00aa980 2024-08-15T17:58:43.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.258+0000 7fda2138b700 1 -- 172.21.15.38:0/3197313007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1c152460 con 0x7fda1c13ea80 2024-08-15T17:58:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d96de700 1 Processor -- start 2024-08-15T17:58:43.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d96de700 1 -- start start 2024-08-15T17:58:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d96de700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe8d40b14a0 0x7fe8d4157150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d96de700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe8d40ad750 0x7fe8d40adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d96de700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d41580e0 con 0x7fe8d4156d80 2024-08-15T17:58:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d96de700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d41577b0 con 0x7fe8d40b14a0 2024-08-15T17:58:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d96de700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d4157a90 con 0x7fe8d40ad750 2024-08-15T17:58:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d37fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe8d40b14a0 0x7fe8d4157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d37fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe8d40b14a0 0x7fe8d4157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53962/0 (socket says 172.21.15.38:53962) 2024-08-15T17:58:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d37fe700 1 -- 172.21.15.38:0/1950108057 learned_addr learned my addr 172.21.15.38:0/1950108057 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d8edd700 1 --2- 172.21.15.38:0/1950108057 >> v2:172.21.15.104:3300/0 conn(0x7fe8d40ad750 0x7fe8d40adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d37fe700 1 -- 172.21.15.38:0/1950108057 >> v2:172.21.15.104:3300/0 conn(0x7fe8d40ad750 msgr2=0x7fe8d40adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d37fe700 1 --2- 172.21.15.38:0/1950108057 >> v2:172.21.15.104:3300/0 conn(0x7fe8d40ad750 0x7fe8d40adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d37fe700 1 -- 172.21.15.38:0/1950108057 >> v1:172.21.15.104:6789/0 conn(0x7fe8d4156d80 legacy=0x7fe8d40b0d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-15T17:58:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.262+0000 7fe8d37fe700 1 -- 172.21.15.38:0/1950108057 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d4157d70 con 0x7fe8d40b14a0 2024-08-15T17:58:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d37fe700 1 --2- 172.21.15.38:0/1950108057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d40b14a0 0x7fe8d4157150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40c7ef0 tx=0x7fe8c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbc061cc57e70c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d2ffd700 1 -- 172.21.15.38:0/1950108057 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800b820 con 0x7fe8d40b14a0 2024-08-15T17:58:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d2ffd700 1 -- 172.21.15.38:0/1950108057 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c800b980 con 0x7fe8d40b14a0 2024-08-15T17:58:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d2ffd700 1 -- 172.21.15.38:0/1950108057 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800bd20 con 0x7fe8d40b14a0 2024-08-15T17:58:43.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7f14ccff9700 1 -- 172.21.15.38:0/656546076 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f14c004e630 con 0x7f14d00aa980 2024-08-15T17:58:43.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fda03fff700 1 -- 172.21.15.38:0/3197313007 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fda1404c890 con 0x7fda1c13ea80 2024-08-15T17:58:43.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d96de700 1 -- 172.21.15.38:0/1950108057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d40b14a0 msgr2=0x7fe8d4157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d96de700 1 --2- 172.21.15.38:0/1950108057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d40b14a0 0x7fe8d4157150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40c7ef0 tx=0x7fe8c8009250 comp rx=0 tx=0).stop 2024-08-15T17:58:43.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d96de700 1 -- 172.21.15.38:0/1950108057 shutdown_connections 2024-08-15T17:58:43.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d96de700 1 --2- 172.21.15.38:0/1950108057 >> v2:172.21.15.104:3300/0 conn(0x7fe8d40ad750 0x7fe8d40adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d96de700 1 --2- 172.21.15.38:0/1950108057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d40b14a0 0x7fe8d4157150 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d96de700 1 -- 172.21.15.38:0/1950108057 >> 172.21.15.38:0/1950108057 conn(0x7fe8d40678d0 msgr2=0x7fe8d4067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d96de700 1 -- 172.21.15.38:0/1950108057 shutdown_connections 2024-08-15T17:58:43.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.266+0000 7fe8d96de700 1 -- 172.21.15.38:0/1950108057 wait complete. 2024-08-15T17:58:43.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d96de700 1 Processor -- start 2024-08-15T17:58:43.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d96de700 1 -- start start 2024-08-15T17:58:43.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d96de700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8d40ad750 0x7fe8d4175c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d96de700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d40b14a0 0x7fe8d415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d96de700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8d4156d80 0x7fe8d415f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d96de700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415f080 con 0x7fe8d40b14a0 2024-08-15T17:58:43.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d96de700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4176160 con 0x7fe8d40ad750 2024-08-15T17:58:43.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d96de700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415e470 con 0x7fe8d4156d80 2024-08-15T17:58:43.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d8edd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8d4156d80 0x7fe8d415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d8edd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8d4156d80 0x7fe8d415f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:44262/0 (socket says 172.21.15.38:44262) 2024-08-15T17:58:43.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d8edd700 1 -- 172.21.15.38:0/886817723 learned_addr learned my addr 172.21.15.38:0/886817723 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.270+0000 7fe8d37fe700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d40b14a0 0x7fe8d415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.274+0000 7fe8d8edd700 1 -- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8d40ad750 msgr2=0x7fe8d4175c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T17:58:43.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.274+0000 7fe8d8edd700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8d40ad750 0x7fe8d4175c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.274+0000 7fe8d8edd700 1 -- 172.21.15.38:0/886817723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d40b14a0 msgr2=0x7fe8d415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.274+0000 7fe8d8edd700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d40b14a0 0x7fe8d415cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.274+0000 7fe8d8edd700 1 -- 172.21.15.38:0/886817723 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c80094f0 con 0x7fe8d4156d80 2024-08-15T17:58:43.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.274+0000 7fe8d37fe700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d40b14a0 0x7fe8d415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:43.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.274+0000 7fe8d8edd700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8d4156d80 0x7fe8d415f510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009fa0 tx=0x7fe8cc009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.278+0000 7fe8bbfff700 1 -- 172.21.15.38:0/886817723 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00ec70 con 0x7fe8d4156d80 2024-08-15T17:58:43.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.278+0000 7fe8bbfff700 1 -- 172.21.15.38:0/886817723 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc00edd0 con 0x7fe8d4156d80 2024-08-15T17:58:43.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.278+0000 7fe8bbfff700 1 -- 172.21.15.38:0/886817723 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc014670 con 0x7fe8d4156d80 2024-08-15T17:58:43.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.278+0000 7fe8d96de700 1 -- 172.21.15.38:0/886817723 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d415e750 con 0x7fe8d4156d80 2024-08-15T17:58:43.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.278+0000 7fe8d96de700 1 -- 172.21.15.38:0/886817723 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d415ed10 con 0x7fe8d4156d80 2024-08-15T17:58:43.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.278+0000 7fe8bbfff700 1 -- 172.21.15.38:0/886817723 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe8cc0149e0 con 0x7fe8d4156d80 2024-08-15T17:58:43.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.278+0000 7fe8bbfff700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe8c0060930 0x7fe8c0062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.278+0000 7fe8bbfff700 1 -- 172.21.15.38:0/886817723 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe8cc07d230 con 0x7fe8d4156d80 2024-08-15T17:58:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.278+0000 7fe8d3fff700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe8c0060930 0x7fe8c0062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.282+0000 7fe8d3fff700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe8c0060930 0x7fe8c0062de0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40096a0 tx=0x7fe8c4006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.286+0000 7fe8b9ffb700 1 -- 172.21.15.38:0/886817723 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d40643a0 con 0x7fe8d4156d80 2024-08-15T17:58:43.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.290+0000 7fe8bbfff700 1 -- 172.21.15.38:0/886817723 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe8cc04d700 con 0x7fe8d4156d80 2024-08-15T17:58:43.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.454+0000 7f6ad5ffb700 1 -- 172.21.15.38:0/2518165544 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6ac4005560 con 0x7f6ae813b630 2024-08-15T17:58:43.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.458+0000 7f6ad7fff700 1 -- 172.21.15.38:0/2518165544 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6ae0056070 con 0x7f6ae813b630 2024-08-15T17:58:43.461 INFO:teuthology.orchestra.run.smithi038.stdout:60129542150 2024-08-15T17:58:43.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 -- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6acc0609b0 msgr2=0x7f6acc062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6acc0609b0 0x7f6acc062e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009f60 tx=0x7f6ad80091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 -- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ae813b630 msgr2=0x7f6ae813a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ae813b630 0x7f6ae813a590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0017000 tx=0x7f6ae001a000 comp rx=0 tx=0).stop 2024-08-15T17:58:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 -- 172.21.15.38:0/2518165544 shutdown_connections 2024-08-15T17:58:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6acc0609b0 0x7f6acc062e60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ae813bf10 0x7f6ae81653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ae813b630 0x7f6ae813a590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 --2- 172.21.15.38:0/2518165544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ae80b1950 0x7f6ae81565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 -- 172.21.15.38:0/2518165544 >> 172.21.15.38:0/2518165544 conn(0x7f6ae80678d0 msgr2=0x7f6ae8069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 -- 172.21.15.38:0/2518165544 shutdown_connections 2024-08-15T17:58:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.466+0000 7f6aee490700 1 -- 172.21.15.38:0/2518165544 wait complete. 2024-08-15T17:58:43.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-08-15T17:58:43.482 DEBUG:teuthology.parallel:result is None 2024-08-15T17:58:43.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.506+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff4ac06f5f0 con 0x7ff4ac06e620 2024-08-15T17:58:43.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.506+0000 7ff4a8ff9700 1 -- 172.21.15.38:0/415533037 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4a401a020 con 0x7ff4ac06e620 2024-08-15T17:58:43.510 INFO:teuthology.orchestra.run.smithi038.stdout:64424509447 2024-08-15T17:58:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff498060860 msgr2=0x7ff498062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff498060860 0x7ff498062d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac068a80 tx=0x7ff4a0009040 comp rx=0 tx=0).stop 2024-08-15T17:58:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4ac06e620 msgr2=0x7ff4ac16cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4ac06e620 0x7ff4ac16cae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400bbb0 tx=0x7ff4a4009840 comp rx=0 tx=0).stop 2024-08-15T17:58:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 shutdown_connections 2024-08-15T17:58:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff498060860 0x7ff498062d10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4ac15d910 0x7ff4ac0a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4ac158fb0 0x7ff4ac1477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 --2- 172.21.15.38:0/415533037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4ac06e620 0x7ff4ac16cae0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 >> 172.21.15.38:0/415533037 conn(0x7ff4ac0678d0 msgr2=0x7ff4ac06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 shutdown_connections 2024-08-15T17:58:43.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.510+0000 7ff4b1e4c700 1 -- 172.21.15.38:0/415533037 wait complete. 2024-08-15T17:58:43.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.514+0000 7f96f106c700 1 -- 172.21.15.38:0/2342586702 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f96ec0b0c60 con 0x7f96ec158fb0 2024-08-15T17:58:43.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.514+0000 7f96e3fff700 1 -- 172.21.15.38:0/2342586702 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f96e40525c0 con 0x7f96ec158fb0 2024-08-15T17:58:43.517 INFO:teuthology.orchestra.run.smithi038.stdout:60129542151 2024-08-15T17:58:43.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 -- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f96cc060930 msgr2=0x7f96cc062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f96cc060930 0x7f96cc062de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f96d4011480 tx=0x7f96d40091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 -- 172.21.15.38:0/2342586702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ec158fb0 msgr2=0x7f96ec176c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ec158fb0 0x7f96ec176c30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96ec06beb0 tx=0x7f96e4018000 comp rx=0 tx=0).stop 2024-08-15T17:58:43.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 -- 172.21.15.38:0/2342586702 shutdown_connections 2024-08-15T17:58:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f96cc060930 0x7f96cc062de0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96ec15d910 0x7f96ec1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96ec158fb0 0x7f96ec176c30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 --2- 172.21.15.38:0/2342586702 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96ec06e450 0x7f96ec179bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 -- 172.21.15.38:0/2342586702 >> 172.21.15.38:0/2342586702 conn(0x7f96ec0678d0 msgr2=0x7f96ec0aa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 -- 172.21.15.38:0/2342586702 shutdown_connections 2024-08-15T17:58:43.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.518+0000 7f96f106c700 1 -- 172.21.15.38:0/2342586702 wait complete. 2024-08-15T17:58:43.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-08-15T17:58:43.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.526+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f411c06f5f0 con 0x7f411c13ea80 2024-08-15T17:58:43.529 DEBUG:teuthology.parallel:result is None 2024-08-15T17:58:43.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.526+0000 7f410a7fc700 1 -- 172.21.15.38:0/4024675295 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4110050230 con 0x7f411c13ea80 2024-08-15T17:58:43.530 INFO:teuthology.orchestra.run.smithi038.stdout:60129542151 2024-08-15T17:58:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f40fc060a60 msgr2=0x7f40fc062f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f40fc060a60 0x7f40fc062f10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f411c068640 tx=0x7f410c00b430 comp rx=0 tx=0).stop 2024-08-15T17:58:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f411c13ea80 msgr2=0x7f411c154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f411c13ea80 0x7f411c154400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f411000b5e0 tx=0x7f41100090f0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 shutdown_connections 2024-08-15T17:58:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f40fc060a60 0x7f40fc062f10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f411c0c0de0 0x7f411c0c1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f411c13ea80 0x7f411c154400 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 --2- 172.21.15.38:0/4024675295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f411c13e6b0 0x7f411c1363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 >> 172.21.15.38:0/4024675295 conn(0x7f411c0678d0 msgr2=0x7f411c0c8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 shutdown_connections 2024-08-15T17:58:43.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.530+0000 7f411b59e700 1 -- 172.21.15.38:0/4024675295 wait complete. 2024-08-15T17:58:43.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-08-15T17:58:43.534 DEBUG:teuthology.parallel:result is None 2024-08-15T17:58:43.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.534+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8bdc005830 con 0x7f8bfc06e570 2024-08-15T17:58:43.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.534+0000 7f8bf8ff9700 1 -- 172.21.15.38:0/1035202108 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8bf4052b30 con 0x7f8bfc06e570 2024-08-15T17:58:43.536 INFO:teuthology.orchestra.run.smithi038.stdout:60129542150 2024-08-15T17:58:43.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8be8060930 msgr2=0x7f8be8062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8be8060930 0x7f8be8062de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0003b50 tx=0x7f8bf0007e20 comp rx=0 tx=0).stop 2024-08-15T17:58:43.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bfc06e570 msgr2=0x7f8bfc161610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bfc06e570 0x7f8bfc161610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc06beb0 tx=0x7f8bf4018000 comp rx=0 tx=0).stop 2024-08-15T17:58:43.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 shutdown_connections 2024-08-15T17:58:43.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8be8060930 0x7f8be8062de0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8bfc0af190 0x7f8bfc161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bfc06e570 0x7f8bfc161610 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 --2- 172.21.15.38:0/1035202108 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8bfc06e1a0 0x7f8bfc086a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 >> 172.21.15.38:0/1035202108 conn(0x7f8bfc0678d0 msgr2=0x7f8bfc15c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 shutdown_connections 2024-08-15T17:58:43.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.538+0000 7f8c01f3e700 1 -- 172.21.15.38:0/1035202108 wait complete. 2024-08-15T17:58:43.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.5 2024-08-15T17:58:43.542 DEBUG:teuthology.parallel:result is None 2024-08-15T17:58:43.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 -- 172.21.15.38:0/3197313007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fda1c0643a0 con 0x7fda1c13ea80 2024-08-15T17:58:43.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda03fff700 1 -- 172.21.15.38:0/3197313007 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fda14018030 con 0x7fda1c13ea80 2024-08-15T17:58:43.547 INFO:teuthology.orchestra.run.smithi038.stdout:64424509448 2024-08-15T17:58:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 -- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fda080609b0 msgr2=0x7fda08062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fda080609b0 0x7fda08062e60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fda1c13d9a0 tx=0x7fda0c009040 comp rx=0 tx=0).stop 2024-08-15T17:58:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 -- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda1c13ea80 msgr2=0x7fda1c12ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda1c13ea80 0x7fda1c12ed00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fda14001c80 tx=0x7fda140061b0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 -- 172.21.15.38:0/3197313007 shutdown_connections 2024-08-15T17:58:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fda080609b0 0x7fda08062e60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda1c12bfe0 0x7fda1c0ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda1c13ea80 0x7fda1c12ed00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 --2- 172.21.15.38:0/3197313007 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda1c13e6b0 0x7fda1c152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 -- 172.21.15.38:0/3197313007 >> 172.21.15.38:0/3197313007 conn(0x7fda1c0678d0 msgr2=0x7fda1c06e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 -- 172.21.15.38:0/3197313007 shutdown_connections 2024-08-15T17:58:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.546+0000 7fda2138b700 1 -- 172.21.15.38:0/3197313007 wait complete. 2024-08-15T17:58:43.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2024-08-15T17:58:43.552 DEBUG:teuthology.parallel:result is None 2024-08-15T17:58:43.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.7 2024-08-15T17:58:43.559 DEBUG:teuthology.parallel:result is None 2024-08-15T17:58:43.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.558+0000 7f14be7fc700 1 -- 172.21.15.38:0/656546076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f14d0070360 con 0x7f14d00aa980 2024-08-15T17:58:43.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.558+0000 7f14ccff9700 1 -- 172.21.15.38:0/656546076 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14c0015020 con 0x7f14d00aa980 2024-08-15T17:58:43.560 INFO:teuthology.orchestra.run.smithi038.stdout:64424509447 2024-08-15T17:58:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.558+0000 7f14be7fc700 1 -- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f14b4060860 msgr2=0x7f14b4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.558+0000 7f14be7fc700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f14b4060860 0x7f14b4062d10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14d0099a90 tx=0x7f14c40091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.558+0000 7f14be7fc700 1 -- 172.21.15.38:0/656546076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d00aa980 msgr2=0x7f14d00b0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.558+0000 7f14be7fc700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d00aa980 0x7f14d00b0450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14c0016000 tx=0x7f14c0017000 comp rx=0 tx=0).stop 2024-08-15T17:58:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.562+0000 7f14be7fc700 1 -- 172.21.15.38:0/656546076 shutdown_connections 2024-08-15T17:58:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.562+0000 7f14be7fc700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f14b4060860 0x7f14b4062d10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.562+0000 7f14be7fc700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14d015a8b0 0x7f14d0156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.562+0000 7f14be7fc700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14d01593b0 0x7f14d00afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.562+0000 7f14be7fc700 1 --2- 172.21.15.38:0/656546076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d00aa980 0x7f14d00b0450 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.562+0000 7f14be7fc700 1 -- 172.21.15.38:0/656546076 >> 172.21.15.38:0/656546076 conn(0x7f14d0068300 msgr2=0x7f14d006e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.562+0000 7f14be7fc700 1 -- 172.21.15.38:0/656546076 shutdown_connections 2024-08-15T17:58:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.562+0000 7f14be7fc700 1 -- 172.21.15.38:0/656546076 wait complete. 2024-08-15T17:58:43.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-08-15T17:58:43.574 DEBUG:teuthology.parallel:result is None 2024-08-15T17:58:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.578+0000 7fe8b9ffb700 1 -- 172.21.15.38:0/886817723 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe8d406f5f0 con 0x7fe8d4156d80 2024-08-15T17:58:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.578+0000 7fe8bbfff700 1 -- 172.21.15.38:0/886817723 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe8cc050d20 con 0x7fe8d4156d80 2024-08-15T17:58:43.582 INFO:teuthology.orchestra.run.smithi038.stdout:60129542152 2024-08-15T17:58:43.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 -- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe8c0060930 msgr2=0x7fe8c0062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe8c0060930 0x7fe8c0062de0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40096a0 tx=0x7fe8c4006bc0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 -- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8d4156d80 msgr2=0x7fe8d415f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8d4156d80 0x7fe8d415f510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009fa0 tx=0x7fe8cc009900 comp rx=0 tx=0).stop 2024-08-15T17:58:43.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 -- 172.21.15.38:0/886817723 shutdown_connections 2024-08-15T17:58:43.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe8c0060930 0x7fe8c0062de0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8d4156d80 0x7fe8d415f510 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d40b14a0 0x7fe8d415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 --2- 172.21.15.38:0/886817723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8d40ad750 0x7fe8d4175c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 -- 172.21.15.38:0/886817723 >> 172.21.15.38:0/886817723 conn(0x7fe8d40678d0 msgr2=0x7fe8d406a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 -- 172.21.15.38:0/886817723 shutdown_connections 2024-08-15T17:58:43.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.582+0000 7fe8b9ffb700 1 -- 172.21.15.38:0/886817723 wait complete. 2024-08-15T17:58:43.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.3 2024-08-15T17:58:43.593 DEBUG:teuthology.parallel:result is None 2024-08-15T17:58:43.593 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-15T17:58:43.593 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:58:43.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.738+0000 7f54917c0700 1 Processor -- start 2024-08-15T17:58:43.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 -- start start 2024-08-15T17:58:43.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c0ad1b0 0x7f548c0b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f548c15cd90 0x7f548c0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c0c86e0 con 0x7f548c15c620 2024-08-15T17:58:43.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548c0cb850 con 0x7f548c15cd90 2024-08-15T17:58:43.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c15d1c0 con 0x7f548c0ad1b0 2024-08-15T17:58:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f548b7fe700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c0ad1b0 0x7f548c0b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f5490fbf700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f548c15cd90 0x7f548c0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f548b7fe700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c0ad1b0 0x7f548c0b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:44270/0 (socket says 172.21.15.38:44270) 2024-08-15T17:58:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f548b7fe700 1 -- 172.21.15.38:0/3832711781 learned_addr learned my addr 172.21.15.38:0/3832711781 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f548b7fe700 1 -- 172.21.15.38:0/3832711781 >> v2:172.21.15.104:3300/0 conn(0x7f548c15cd90 msgr2=0x7f548c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f548b7fe700 1 --2- 172.21.15.38:0/3832711781 >> v2:172.21.15.104:3300/0 conn(0x7f548c15cd90 0x7f548c0c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f548b7fe700 1 -- 172.21.15.38:0/3832711781 >> v1:172.21.15.104:6789/0 conn(0x7f548c15c620 legacy=0x7f548c0acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f548b7fe700 1 -- 172.21.15.38:0/3832711781 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c15d490 con 0x7f548c0ad1b0 2024-08-15T17:58:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f5490fbf700 1 --2- 172.21.15.38:0/3832711781 >> v2:172.21.15.104:3300/0 conn(0x7f548c15cd90 0x7f548c0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f548b7fe700 1 --2- 172.21.15.38:0/3832711781 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c0ad1b0 0x7f548c0b0320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5474009f20 tx=0x7f5474009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4f5225ebf755345 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f548affd700 1 -- 172.21.15.38:0/3832711781 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547400e070 con 0x7f548c0ad1b0 2024-08-15T17:58:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f548affd700 1 -- 172.21.15.38:0/3832711781 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547400b9d0 con 0x7f548c0ad1b0 2024-08-15T17:58:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f548affd700 1 -- 172.21.15.38:0/3832711781 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54740092e0 con 0x7f548c0ad1b0 2024-08-15T17:58:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 -- 172.21.15.38:0/3832711781 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c0ad1b0 msgr2=0x7f548c0b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 --2- 172.21.15.38:0/3832711781 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c0ad1b0 0x7f548c0b0320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5474009f20 tx=0x7f5474009840 comp rx=0 tx=0).stop 2024-08-15T17:58:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 -- 172.21.15.38:0/3832711781 shutdown_connections 2024-08-15T17:58:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 --2- 172.21.15.38:0/3832711781 >> v2:172.21.15.104:3300/0 conn(0x7f548c15cd90 0x7f548c0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 --2- 172.21.15.38:0/3832711781 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c0ad1b0 0x7f548c0b0320 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 -- 172.21.15.38:0/3832711781 >> 172.21.15.38:0/3832711781 conn(0x7f548c0ca900 msgr2=0x7f548c0cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 -- 172.21.15.38:0/3832711781 shutdown_connections 2024-08-15T17:58:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.742+0000 7f54917c0700 1 -- 172.21.15.38:0/3832711781 wait complete. 2024-08-15T17:58:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f54917c0700 1 Processor -- start 2024-08-15T17:58:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f54917c0700 1 -- start start 2024-08-15T17:58:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f54917c0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f548c0ad1b0 0x7f548c14a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f54917c0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f548c15c620 0x7f548c169780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f54917c0700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c15cd90 0x7f548c169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f54917c0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0c7820 con 0x7f548c15c620 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f54917c0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0c5590 con 0x7f548c0ad1b0 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f54917c0700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0c7160 con 0x7f548c15cd90 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f548b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f548c15c620 0x7f548c169780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f548b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f548c15c620 0x7f548c169780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53974/0 (socket says 172.21.15.38:53974) 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f548b7fe700 1 -- 172.21.15.38:0/3784173736 learned_addr learned my addr 172.21.15.38:0/3784173736 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f5490fbf700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c15cd90 0x7f548c169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f548bfff700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f548c0ad1b0 0x7f548c14a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f548b7fe700 1 -- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c15cd90 msgr2=0x7f548c169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f548b7fe700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c15cd90 0x7f548c169cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f548b7fe700 1 -- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f548c0ad1b0 msgr2=0x7f548c14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f548b7fe700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f548c0ad1b0 0x7f548c14a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f548b7fe700 1 -- 172.21.15.38:0/3784173736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484009420 con 0x7f548c15c620 2024-08-15T17:58:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f548b7fe700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f548c15c620 0x7f548c169780 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f547400b610 tx=0x7f5474018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f5483fff700 1 -- 172.21.15.38:0/3784173736 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474008a50 con 0x7f548c15c620 2024-08-15T17:58:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f5483fff700 1 -- 172.21.15.38:0/3784173736 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5474008bb0 con 0x7f548c15c620 2024-08-15T17:58:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f5483fff700 1 -- 172.21.15.38:0/3784173736 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474026460 con 0x7f548c15c620 2024-08-15T17:58:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f54917c0700 1 -- 172.21.15.38:0/3784173736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5474009ae0 con 0x7f548c15c620 2024-08-15T17:58:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.746+0000 7f54917c0700 1 -- 172.21.15.38:0/3784173736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c0c4f40 con 0x7f548c15c620 2024-08-15T17:58:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.750+0000 7f54917c0700 1 -- 172.21.15.38:0/3784173736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c0c5700 con 0x7f548c15c620 2024-08-15T17:58:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.750+0000 7f5483fff700 1 -- 172.21.15.38:0/3784173736 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5474003040 con 0x7f548c15c620 2024-08-15T17:58:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.750+0000 7f5483fff700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f546c060860 0x7f546c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.750+0000 7f5483fff700 1 -- 172.21.15.38:0/3784173736 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f547407e8b0 con 0x7f548c15c620 2024-08-15T17:58:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.750+0000 7f548bfff700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f546c060860 0x7f546c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.750+0000 7f548bfff700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f546c060860 0x7f546c062d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f547c003860 tx=0x7f547c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.758+0000 7f5483fff700 1 -- 172.21.15.38:0/3784173736 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f547404edc0 con 0x7f548c15c620 2024-08-15T17:58:43.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.910+0000 7f54917c0700 1 -- 172.21.15.38:0/3784173736 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f548c0643a0 con 0x7f546c060860 2024-08-15T17:58:43.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.926+0000 7f5483fff700 1 -- 172.21.15.38:0/3784173736 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264525 (secure 0 0 0) 0x7f548c0643a0 con 0x7f546c060860 2024-08-15T17:58:43.931 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:43.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 -- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f546c060860 msgr2=0x7f546c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f546c060860 0x7f546c062d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f547c003860 tx=0x7f547c009040 comp rx=0 tx=0).stop 2024-08-15T17:58:43.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 -- 172.21.15.38:0/3784173736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f548c15c620 msgr2=0x7f548c169780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:43.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f548c15c620 0x7f548c169780 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f547400b610 tx=0x7f5474018000 comp rx=0 tx=0).stop 2024-08-15T17:58:43.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 -- 172.21.15.38:0/3784173736 shutdown_connections 2024-08-15T17:58:43.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f546c060860 0x7f546c062d10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f548c15cd90 0x7f548c169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f548c15c620 0x7f548c169780 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 --2- 172.21.15.38:0/3784173736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f548c0ad1b0 0x7f548c14a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:43.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 -- 172.21.15.38:0/3784173736 >> 172.21.15.38:0/3784173736 conn(0x7f548c0ca900 msgr2=0x7f548c0ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:43.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 -- 172.21.15.38:0/3784173736 shutdown_connections 2024-08-15T17:58:43.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:43.930+0000 7f54917c0700 1 -- 172.21.15.38:0/3784173736 wait complete. 2024-08-15T17:58:43.934 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-08-15T17:58:43.946 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-08-15T17:58:42.169638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2390044,"kb_used_data":2856,"kb_used_omap":13,"kb_used_meta":2387058,"kb_avail":752584676,"statfs":{"total":773094113280,"available":770646708224,"internally_reserved":0,"allocated":2924544,"data_stored":1767820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2444348096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.923497"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.295385+0000","last_change":"2024-08-15T17:58:36.292335+0000","last_active":"2024-08-15T17:58:36.295385+0000","last_peered":"2024-08-15T17:58:36.295385+0000","last_clean":"2024-08-15T17:58:36.295385+0000","last_became_active":"2024-08-15T17:58:36.292033+0000","last_became_peered":"2024-08-15T17:58:36.292033+0000","last_unstale":"2024-08-15T17:58:36.295385+0000","last_undegraded":"2024-08-15T17:58:36.295385+0000","last_fullsized":"2024-08-15T17:58:36.295385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:59:24.721036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289389+0000","last_change":"2024-08-15T17:58:36.286540+0000","last_active":"2024-08-15T17:58:36.289389+0000","last_peered":"2024-08-15T17:58:36.289389+0000","last_clean":"2024-08-15T17:58:36.289389+0000","last_became_active":"2024-08-15T17:58:36.285657+0000","last_became_peered":"2024-08-15T17:58:36.285657+0000","last_unstale":"2024-08-15T17:58:36.289389+0000","last_undegraded":"2024-08-15T17:58:36.289389+0000","last_fullsized":"2024-08-15T17:58:36.289389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:50:39.313237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288210+0000","last_change":"2024-08-15T17:58:36.279817+0000","last_active":"2024-08-15T17:58:36.288210+0000","last_peered":"2024-08-15T17:58:36.288210+0000","last_clean":"2024-08-15T17:58:36.288210+0000","last_became_active":"2024-08-15T17:58:36.279494+0000","last_became_peered":"2024-08-15T17:58:36.279494+0000","last_unstale":"2024-08-15T17:58:36.288210+0000","last_undegraded":"2024-08-15T17:58:36.288210+0000","last_fullsized":"2024-08-15T17:58:36.288210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:48:29.832597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288855+0000","last_change":"2024-08-15T17:58:36.285873+0000","last_active":"2024-08-15T17:58:36.288855+0000","last_peered":"2024-08-15T17:58:36.288855+0000","last_clean":"2024-08-15T17:58:36.288855+0000","last_became_active":"2024-08-15T17:58:36.285630+0000","last_became_peered":"2024-08-15T17:58:36.285630+0000","last_unstale":"2024-08-15T17:58:36.288855+0000","last_undegraded":"2024-08-15T17:58:36.288855+0000","last_fullsized":"2024-08-15T17:58:36.288855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:30:51.740686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.291564+0000","last_change":"2024-08-15T17:58:36.281869+0000","last_active":"2024-08-15T17:58:36.291564+0000","last_peered":"2024-08-15T17:58:36.291564+0000","last_clean":"2024-08-15T17:58:36.291564+0000","last_became_active":"2024-08-15T17:58:36.281550+0000","last_became_peered":"2024-08-15T17:58:36.281550+0000","last_unstale":"2024-08-15T17:58:36.291564+0000","last_undegraded":"2024-08-15T17:58:36.291564+0000","last_fullsized":"2024-08-15T17:58:36.291564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:33:16.117022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288868+0000","last_change":"2024-08-15T17:58:36.286578+0000","last_active":"2024-08-15T17:58:36.288868+0000","last_peered":"2024-08-15T17:58:36.288868+0000","last_clean":"2024-08-15T17:58:36.288868+0000","last_became_active":"2024-08-15T17:58:36.286298+0000","last_became_peered":"2024-08-15T17:58:36.286298+0000","last_unstale":"2024-08-15T17:58:36.288868+0000","last_undegraded":"2024-08-15T17:58:36.288868+0000","last_fullsized":"2024-08-15T17:58:36.288868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:47:17.486762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.295000+0000","last_change":"2024-08-15T17:58:36.286906+0000","last_active":"2024-08-15T17:58:36.295000+0000","last_peered":"2024-08-15T17:58:36.295000+0000","last_clean":"2024-08-15T17:58:36.295000+0000","last_became_active":"2024-08-15T17:58:36.286620+0000","last_became_peered":"2024-08-15T17:58:36.286620+0000","last_unstale":"2024-08-15T17:58:36.295000+0000","last_undegraded":"2024-08-15T17:58:36.295000+0000","last_fullsized":"2024-08-15T17:58:36.295000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:35:39.618955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.287670+0000","last_change":"2024-08-15T17:58:36.285830+0000","last_active":"2024-08-15T17:58:36.287670+0000","last_peered":"2024-08-15T17:58:36.287670+0000","last_clean":"2024-08-15T17:58:36.287670+0000","last_became_active":"2024-08-15T17:58:36.285422+0000","last_became_peered":"2024-08-15T17:58:36.285422+0000","last_unstale":"2024-08-15T17:58:36.287670+0000","last_undegraded":"2024-08-15T17:58:36.287670+0000","last_fullsized":"2024-08-15T17:58:36.287670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:15:59.759500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.295346+0000","last_change":"2024-08-15T17:58:36.286462+0000","last_active":"2024-08-15T17:58:36.295346+0000","last_peered":"2024-08-15T17:58:36.295346+0000","last_clean":"2024-08-15T17:58:36.295346+0000","last_became_active":"2024-08-15T17:58:36.285671+0000","last_became_peered":"2024-08-15T17:58:36.285671+0000","last_unstale":"2024-08-15T17:58:36.295346+0000","last_undegraded":"2024-08-15T17:58:36.295346+0000","last_fullsized":"2024-08-15T17:58:36.295346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:22:51.492443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294455+0000","last_change":"2024-08-15T17:58:36.289845+0000","last_active":"2024-08-15T17:58:36.294455+0000","last_peered":"2024-08-15T17:58:36.294455+0000","last_clean":"2024-08-15T17:58:36.294455+0000","last_became_active":"2024-08-15T17:58:36.286963+0000","last_became_peered":"2024-08-15T17:58:36.286963+0000","last_unstale":"2024-08-15T17:58:36.294455+0000","last_undegraded":"2024-08-15T17:58:36.294455+0000","last_fullsized":"2024-08-15T17:58:36.294455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:49:19.919158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.295045+0000","last_change":"2024-08-15T17:58:36.292046+0000","last_active":"2024-08-15T17:58:36.295045+0000","last_peered":"2024-08-15T17:58:36.295045+0000","last_clean":"2024-08-15T17:58:36.295045+0000","last_became_active":"2024-08-15T17:58:36.291772+0000","last_became_peered":"2024-08-15T17:58:36.291772+0000","last_unstale":"2024-08-15T17:58:36.295045+0000","last_undegraded":"2024-08-15T17:58:36.295045+0000","last_fullsized":"2024-08-15T17:58:36.295045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:11:45.354221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.291455+0000","last_change":"2024-08-15T17:58:36.288242+0000","last_active":"2024-08-15T17:58:36.291455+0000","last_peered":"2024-08-15T17:58:36.291455+0000","last_clean":"2024-08-15T17:58:36.291455+0000","last_became_active":"2024-08-15T17:58:36.287991+0000","last_became_peered":"2024-08-15T17:58:36.287991+0000","last_unstale":"2024-08-15T17:58:36.291455+0000","last_undegraded":"2024-08-15T17:58:36.291455+0000","last_fullsized":"2024-08-15T17:58:36.291455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:11:27.250315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.292602+0000","last_change":"2024-08-15T17:58:36.286478+0000","last_active":"2024-08-15T17:58:36.292602+0000","last_peered":"2024-08-15T17:58:36.292602+0000","last_clean":"2024-08-15T17:58:36.292602+0000","last_became_active":"2024-08-15T17:58:36.285967+0000","last_became_peered":"2024-08-15T17:58:36.285967+0000","last_unstale":"2024-08-15T17:58:36.292602+0000","last_undegraded":"2024-08-15T17:58:36.292602+0000","last_fullsized":"2024-08-15T17:58:36.292602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288295+0000","last_change":"2024-08-15T17:58:36.279946+0000","last_active":"2024-08-15T17:58:36.288295+0000","last_peered":"2024-08-15T17:58:36.288295+0000","last_clean":"2024-08-15T17:58:36.288295+0000","last_became_active":"2024-08-15T17:58:36.279648+0000","last_became_peered":"2024-08-15T17:58:36.279648+0000","last_unstale":"2024-08-15T17:58:36.288295+0000","last_undegraded":"2024-08-15T17:58:36.288295+0000","last_fullsized":"2024-08-15T17:58:36.288295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:34:14.015831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289440+0000","last_change":"2024-08-15T17:58:36.286890+0000","last_active":"2024-08-15T17:58:36.289440+0000","last_peered":"2024-08-15T17:58:36.289440+0000","last_clean":"2024-08-15T17:58:36.289440+0000","last_became_active":"2024-08-15T17:58:36.286332+0000","last_became_peered":"2024-08-15T17:58:36.286332+0000","last_unstale":"2024-08-15T17:58:36.289440+0000","last_undegraded":"2024-08-15T17:58:36.289440+0000","last_fullsized":"2024-08-15T17:58:36.289440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:07:28.572500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293942+0000","last_change":"2024-08-15T17:58:36.287994+0000","last_active":"2024-08-15T17:58:36.293942+0000","last_peered":"2024-08-15T17:58:36.293942+0000","last_clean":"2024-08-15T17:58:36.293942+0000","last_became_active":"2024-08-15T17:58:36.286645+0000","last_became_peered":"2024-08-15T17:58:36.286645+0000","last_unstale":"2024-08-15T17:58:36.293942+0000","last_undegraded":"2024-08-15T17:58:36.293942+0000","last_fullsized":"2024-08-15T17:58:36.293942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285495+0000","last_change":"2024-08-15T17:58:36.280114+0000","last_active":"2024-08-15T17:58:36.285495+0000","last_peered":"2024-08-15T17:58:36.285495+0000","last_clean":"2024-08-15T17:58:36.285495+0000","last_became_active":"2024-08-15T17:58:36.279785+0000","last_became_peered":"2024-08-15T17:58:36.279785+0000","last_unstale":"2024-08-15T17:58:36.285495+0000","last_undegraded":"2024-08-15T17:58:36.285495+0000","last_fullsized":"2024-08-15T17:58:36.285495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:56:03.150852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.292089+0000","last_change":"2024-08-15T17:58:36.290521+0000","last_active":"2024-08-15T17:58:36.292089+0000","last_peered":"2024-08-15T17:58:36.292089+0000","last_clean":"2024-08-15T17:58:36.292089+0000","last_became_active":"2024-08-15T17:58:36.290154+0000","last_became_peered":"2024-08-15T17:58:36.290154+0000","last_unstale":"2024-08-15T17:58:36.292089+0000","last_undegraded":"2024-08-15T17:58:36.292089+0000","last_fullsized":"2024-08-15T17:58:36.292089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:45.183805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.296070+0000","last_change":"2024-08-15T17:58:36.285210+0000","last_active":"2024-08-15T17:58:36.296070+0000","last_peered":"2024-08-15T17:58:36.296070+0000","last_clean":"2024-08-15T17:58:36.296070+0000","last_became_active":"2024-08-15T17:58:36.284752+0000","last_became_peered":"2024-08-15T17:58:36.284752+0000","last_unstale":"2024-08-15T17:58:36.296070+0000","last_undegraded":"2024-08-15T17:58:36.296070+0000","last_fullsized":"2024-08-15T17:58:36.296070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:18:07.754382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.292174+0000","last_change":"2024-08-15T17:58:36.285554+0000","last_active":"2024-08-15T17:58:36.292174+0000","last_peered":"2024-08-15T17:58:36.292174+0000","last_clean":"2024-08-15T17:58:36.292174+0000","last_became_active":"2024-08-15T17:58:36.285281+0000","last_became_peered":"2024-08-15T17:58:36.285281+0000","last_unstale":"2024-08-15T17:58:36.292174+0000","last_undegraded":"2024-08-15T17:58:36.292174+0000","last_fullsized":"2024-08-15T17:58:36.292174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:12:39.229320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290783+0000","last_change":"2024-08-15T17:58:36.280839+0000","last_active":"2024-08-15T17:58:36.290783+0000","last_peered":"2024-08-15T17:58:36.290783+0000","last_clean":"2024-08-15T17:58:36.290783+0000","last_became_active":"2024-08-15T17:58:36.280519+0000","last_became_peered":"2024-08-15T17:58:36.280519+0000","last_unstale":"2024-08-15T17:58:36.290783+0000","last_undegraded":"2024-08-15T17:58:36.290783+0000","last_fullsized":"2024-08-15T17:58:36.290783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.281236+0000","last_change":"2024-08-15T17:58:36.278583+0000","last_active":"2024-08-15T17:58:36.281236+0000","last_peered":"2024-08-15T17:58:36.281236+0000","last_clean":"2024-08-15T17:58:36.281236+0000","last_became_active":"2024-08-15T17:58:36.278308+0000","last_became_peered":"2024-08-15T17:58:36.278308+0000","last_unstale":"2024-08-15T17:58:36.281236+0000","last_undegraded":"2024-08-15T17:58:36.281236+0000","last_fullsized":"2024-08-15T17:58:36.281236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:56:49.069407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286580+0000","last_change":"2024-08-15T17:58:36.283003+0000","last_active":"2024-08-15T17:58:36.286580+0000","last_peered":"2024-08-15T17:58:36.286580+0000","last_clean":"2024-08-15T17:58:36.286580+0000","last_became_active":"2024-08-15T17:58:36.282691+0000","last_became_peered":"2024-08-15T17:58:36.282691+0000","last_unstale":"2024-08-15T17:58:36.286580+0000","last_undegraded":"2024-08-15T17:58:36.286580+0000","last_fullsized":"2024-08-15T17:58:36.286580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:48:28.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288825+0000","last_change":"2024-08-15T17:58:36.285479+0000","last_active":"2024-08-15T17:58:36.288825+0000","last_peered":"2024-08-15T17:58:36.288825+0000","last_clean":"2024-08-15T17:58:36.288825+0000","last_became_active":"2024-08-15T17:58:36.285044+0000","last_became_peered":"2024-08-15T17:58:36.285044+0000","last_unstale":"2024-08-15T17:58:36.288825+0000","last_undegraded":"2024-08-15T17:58:36.288825+0000","last_fullsized":"2024-08-15T17:58:36.288825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:55:54.644188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286608+0000","last_change":"2024-08-15T17:58:36.284044+0000","last_active":"2024-08-15T17:58:36.286608+0000","last_peered":"2024-08-15T17:58:36.286608+0000","last_clean":"2024-08-15T17:58:36.286608+0000","last_became_active":"2024-08-15T17:58:36.283664+0000","last_became_peered":"2024-08-15T17:58:36.283664+0000","last_unstale":"2024-08-15T17:58:36.286608+0000","last_undegraded":"2024-08-15T17:58:36.286608+0000","last_fullsized":"2024-08-15T17:58:36.286608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:58:52.618737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294474+0000","last_change":"2024-08-15T17:58:36.290152+0000","last_active":"2024-08-15T17:58:36.294474+0000","last_peered":"2024-08-15T17:58:36.294474+0000","last_clean":"2024-08-15T17:58:36.294474+0000","last_became_active":"2024-08-15T17:58:36.287149+0000","last_became_peered":"2024-08-15T17:58:36.287149+0000","last_unstale":"2024-08-15T17:58:36.294474+0000","last_undegraded":"2024-08-15T17:58:36.294474+0000","last_fullsized":"2024-08-15T17:58:36.294474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:50:39.313237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289403+0000","last_change":"2024-08-15T17:58:36.287149+0000","last_active":"2024-08-15T17:58:36.289403+0000","last_peered":"2024-08-15T17:58:36.289403+0000","last_clean":"2024-08-15T17:58:36.289403+0000","last_became_active":"2024-08-15T17:58:36.286239+0000","last_became_peered":"2024-08-15T17:58:36.286239+0000","last_unstale":"2024-08-15T17:58:36.289403+0000","last_undegraded":"2024-08-15T17:58:36.289403+0000","last_fullsized":"2024-08-15T17:58:36.289403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:02:13.223698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286778+0000","last_change":"2024-08-15T17:58:36.283315+0000","last_active":"2024-08-15T17:58:36.286778+0000","last_peered":"2024-08-15T17:58:36.286778+0000","last_clean":"2024-08-15T17:58:36.286778+0000","last_became_active":"2024-08-15T17:58:36.282873+0000","last_became_peered":"2024-08-15T17:58:36.282873+0000","last_unstale":"2024-08-15T17:58:36.286778+0000","last_undegraded":"2024-08-15T17:58:36.286778+0000","last_fullsized":"2024-08-15T17:58:36.286778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:31:12.400383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290819+0000","last_change":"2024-08-15T17:58:36.284677+0000","last_active":"2024-08-15T17:58:36.290819+0000","last_peered":"2024-08-15T17:58:36.290819+0000","last_clean":"2024-08-15T17:58:36.290819+0000","last_became_active":"2024-08-15T17:58:36.281734+0000","last_became_peered":"2024-08-15T17:58:36.281734+0000","last_unstale":"2024-08-15T17:58:36.290819+0000","last_undegraded":"2024-08-15T17:58:36.290819+0000","last_fullsized":"2024-08-15T17:58:36.290819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288562+0000","last_change":"2024-08-15T17:58:36.281437+0000","last_active":"2024-08-15T17:58:36.288562+0000","last_peered":"2024-08-15T17:58:36.288562+0000","last_clean":"2024-08-15T17:58:36.288562+0000","last_became_active":"2024-08-15T17:58:36.280874+0000","last_became_peered":"2024-08-15T17:58:36.280874+0000","last_unstale":"2024-08-15T17:58:36.288562+0000","last_undegraded":"2024-08-15T17:58:36.288562+0000","last_fullsized":"2024-08-15T17:58:36.288562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:05.015208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288535+0000","last_change":"2024-08-15T17:58:36.285476+0000","last_active":"2024-08-15T17:58:36.288535+0000","last_peered":"2024-08-15T17:58:36.288535+0000","last_clean":"2024-08-15T17:58:36.288535+0000","last_became_active":"2024-08-15T17:58:36.285049+0000","last_became_peered":"2024-08-15T17:58:36.285049+0000","last_unstale":"2024-08-15T17:58:36.288535+0000","last_undegraded":"2024-08-15T17:58:36.288535+0000","last_fullsized":"2024-08-15T17:58:36.288535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:32:58.557897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294340+0000","last_change":"2024-08-15T17:58:36.286646+0000","last_active":"2024-08-15T17:58:36.294340+0000","last_peered":"2024-08-15T17:58:36.294340+0000","last_clean":"2024-08-15T17:58:36.294340+0000","last_became_active":"2024-08-15T17:58:36.286375+0000","last_became_peered":"2024-08-15T17:58:36.286375+0000","last_unstale":"2024-08-15T17:58:36.294340+0000","last_undegraded":"2024-08-15T17:58:36.294340+0000","last_fullsized":"2024-08-15T17:58:36.294340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:33.556488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285304+0000","last_change":"2024-08-15T17:58:36.278354+0000","last_active":"2024-08-15T17:58:36.285304+0000","last_peered":"2024-08-15T17:58:36.285304+0000","last_clean":"2024-08-15T17:58:36.285304+0000","last_became_active":"2024-08-15T17:58:36.278061+0000","last_became_peered":"2024-08-15T17:58:36.278061+0000","last_unstale":"2024-08-15T17:58:36.285304+0000","last_undegraded":"2024-08-15T17:58:36.285304+0000","last_fullsized":"2024-08-15T17:58:36.285304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:45.183805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301650+0000","last_change":"2024-08-15T17:58:35.277170+0000","last_active":"2024-08-15T17:58:36.301650+0000","last_peered":"2024-08-15T17:58:36.301650+0000","last_clean":"2024-08-15T17:58:36.301650+0000","last_became_active":"2024-08-15T17:58:35.276904+0000","last_became_peered":"2024-08-15T17:58:35.276904+0000","last_unstale":"2024-08-15T17:58:36.301650+0000","last_undegraded":"2024-08-15T17:58:36.301650+0000","last_fullsized":"2024-08-15T17:58:36.301650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:11:34.747972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.274623+0000","last_change":"2024-08-15T17:58:36.271912+0000","last_active":"2024-08-15T17:58:36.274623+0000","last_peered":"2024-08-15T17:58:36.274623+0000","last_clean":"2024-08-15T17:58:36.274623+0000","last_became_active":"2024-08-15T17:58:36.271346+0000","last_became_peered":"2024-08-15T17:58:36.271346+0000","last_unstale":"2024-08-15T17:58:36.274623+0000","last_undegraded":"2024-08-15T17:58:36.274623+0000","last_fullsized":"2024-08-15T17:58:36.274623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:54:02.548772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298247+0000","last_change":"2024-08-15T17:58:35.266446+0000","last_active":"2024-08-15T17:58:36.298247+0000","last_peered":"2024-08-15T17:58:36.298247+0000","last_clean":"2024-08-15T17:58:36.298247+0000","last_became_active":"2024-08-15T17:58:35.265867+0000","last_became_peered":"2024-08-15T17:58:35.265867+0000","last_unstale":"2024-08-15T17:58:36.298247+0000","last_undegraded":"2024-08-15T17:58:36.298247+0000","last_fullsized":"2024-08-15T17:58:36.298247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:11:26.241336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280182+0000","last_change":"2024-08-15T17:58:35.275168+0000","last_active":"2024-08-15T17:58:36.280182+0000","last_peered":"2024-08-15T17:58:36.280182+0000","last_clean":"2024-08-15T17:58:36.280182+0000","last_became_active":"2024-08-15T17:58:35.274863+0000","last_became_peered":"2024-08-15T17:58:35.274863+0000","last_unstale":"2024-08-15T17:58:36.280182+0000","last_undegraded":"2024-08-15T17:58:36.280182+0000","last_fullsized":"2024-08-15T17:58:36.280182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:17:09.213636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293638+0000","last_change":"2024-08-15T17:58:36.290048+0000","last_active":"2024-08-15T17:58:36.293638+0000","last_peered":"2024-08-15T17:58:36.293638+0000","last_clean":"2024-08-15T17:58:36.293638+0000","last_became_active":"2024-08-15T17:58:36.289720+0000","last_became_peered":"2024-08-15T17:58:36.289720+0000","last_unstale":"2024-08-15T17:58:36.293638+0000","last_undegraded":"2024-08-15T17:58:36.293638+0000","last_fullsized":"2024-08-15T17:58:36.293638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:55.518837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.272221+0000","last_change":"2024-08-15T17:58:35.266688+0000","last_active":"2024-08-15T17:58:36.272221+0000","last_peered":"2024-08-15T17:58:36.272221+0000","last_clean":"2024-08-15T17:58:36.272221+0000","last_became_active":"2024-08-15T17:58:35.266306+0000","last_became_peered":"2024-08-15T17:58:35.266306+0000","last_unstale":"2024-08-15T17:58:36.272221+0000","last_undegraded":"2024-08-15T17:58:36.272221+0000","last_fullsized":"2024-08-15T17:58:36.272221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:11:34.747972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289814+0000","last_change":"2024-08-15T17:58:36.287582+0000","last_active":"2024-08-15T17:58:36.289814+0000","last_peered":"2024-08-15T17:58:36.289814+0000","last_clean":"2024-08-15T17:58:36.289814+0000","last_became_active":"2024-08-15T17:58:36.286980+0000","last_became_peered":"2024-08-15T17:58:36.286980+0000","last_unstale":"2024-08-15T17:58:36.289814+0000","last_undegraded":"2024-08-15T17:58:36.289814+0000","last_fullsized":"2024-08-15T17:58:36.289814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:49:19.919158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285974+0000","last_change":"2024-08-15T17:58:35.267834+0000","last_active":"2024-08-15T17:58:36.285974+0000","last_peered":"2024-08-15T17:58:36.285974+0000","last_clean":"2024-08-15T17:58:36.285974+0000","last_became_active":"2024-08-15T17:58:35.267249+0000","last_became_peered":"2024-08-15T17:58:35.267249+0000","last_unstale":"2024-08-15T17:58:36.285974+0000","last_undegraded":"2024-08-15T17:58:36.285974+0000","last_fullsized":"2024-08-15T17:58:36.285974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:17.590038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288210+0000","last_change":"2024-08-15T17:58:36.280965+0000","last_active":"2024-08-15T17:58:36.288210+0000","last_peered":"2024-08-15T17:58:36.288210+0000","last_clean":"2024-08-15T17:58:36.288210+0000","last_became_active":"2024-08-15T17:58:36.280684+0000","last_became_peered":"2024-08-15T17:58:36.280684+0000","last_unstale":"2024-08-15T17:58:36.288210+0000","last_undegraded":"2024-08-15T17:58:36.288210+0000","last_fullsized":"2024-08-15T17:58:36.288210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:15:59.759500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.274432+0000","last_change":"2024-08-15T17:58:35.268261+0000","last_active":"2024-08-15T17:58:36.274432+0000","last_peered":"2024-08-15T17:58:36.274432+0000","last_clean":"2024-08-15T17:58:36.274432+0000","last_became_active":"2024-08-15T17:58:35.266109+0000","last_became_peered":"2024-08-15T17:58:35.266109+0000","last_unstale":"2024-08-15T17:58:36.274432+0000","last_undegraded":"2024-08-15T17:58:36.274432+0000","last_fullsized":"2024-08-15T17:58:36.274432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:06:34.826184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289339+0000","last_change":"2024-08-15T17:58:36.283639+0000","last_active":"2024-08-15T17:58:36.289339+0000","last_peered":"2024-08-15T17:58:36.289339+0000","last_clean":"2024-08-15T17:58:36.289339+0000","last_became_active":"2024-08-15T17:58:36.282343+0000","last_became_peered":"2024-08-15T17:58:36.282343+0000","last_unstale":"2024-08-15T17:58:36.289339+0000","last_undegraded":"2024-08-15T17:58:36.289339+0000","last_fullsized":"2024-08-15T17:58:36.289339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:58:52.618737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301091+0000","last_change":"2024-08-15T17:58:35.276856+0000","last_active":"2024-08-15T17:58:36.301091+0000","last_peered":"2024-08-15T17:58:36.301091+0000","last_clean":"2024-08-15T17:58:36.301091+0000","last_became_active":"2024-08-15T17:58:35.275745+0000","last_became_peered":"2024-08-15T17:58:35.275745+0000","last_unstale":"2024-08-15T17:58:36.301091+0000","last_undegraded":"2024-08-15T17:58:36.301091+0000","last_fullsized":"2024-08-15T17:58:36.301091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:26:33.980860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.279341+0000","last_change":"2024-08-15T17:58:36.272759+0000","last_active":"2024-08-15T17:58:36.279341+0000","last_peered":"2024-08-15T17:58:36.279341+0000","last_clean":"2024-08-15T17:58:36.279341+0000","last_became_active":"2024-08-15T17:58:36.271418+0000","last_became_peered":"2024-08-15T17:58:36.271418+0000","last_unstale":"2024-08-15T17:58:36.279341+0000","last_undegraded":"2024-08-15T17:58:36.279341+0000","last_fullsized":"2024-08-15T17:58:36.279341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:32:58.557897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301498+0000","last_change":"2024-08-15T17:58:35.271651+0000","last_active":"2024-08-15T17:58:36.301498+0000","last_peered":"2024-08-15T17:58:36.301498+0000","last_clean":"2024-08-15T17:58:36.301498+0000","last_became_active":"2024-08-15T17:58:35.271306+0000","last_became_peered":"2024-08-15T17:58:35.271306+0000","last_unstale":"2024-08-15T17:58:36.301498+0000","last_undegraded":"2024-08-15T17:58:36.301498+0000","last_fullsized":"2024-08-15T17:58:36.301498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:44.174826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.296030+0000","last_change":"2024-08-15T17:58:36.284247+0000","last_active":"2024-08-15T17:58:36.296030+0000","last_peered":"2024-08-15T17:58:36.296030+0000","last_clean":"2024-08-15T17:58:36.296030+0000","last_became_active":"2024-08-15T17:58:36.283981+0000","last_became_peered":"2024-08-15T17:58:36.283981+0000","last_unstale":"2024-08-15T17:58:36.296030+0000","last_undegraded":"2024-08-15T17:58:36.296030+0000","last_fullsized":"2024-08-15T17:58:36.296030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:09:08.360709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.305587+0000","last_change":"2024-08-15T17:58:35.268853+0000","last_active":"2024-08-15T17:58:36.305587+0000","last_peered":"2024-08-15T17:58:36.305587+0000","last_clean":"2024-08-15T17:58:36.305587+0000","last_became_active":"2024-08-15T17:58:35.268541+0000","last_became_peered":"2024-08-15T17:58:35.268541+0000","last_unstale":"2024-08-15T17:58:36.305587+0000","last_undegraded":"2024-08-15T17:58:36.305587+0000","last_fullsized":"2024-08-15T17:58:36.305587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:33:31.329931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288269+0000","last_change":"2024-08-15T17:58:36.278321+0000","last_active":"2024-08-15T17:58:36.288269+0000","last_peered":"2024-08-15T17:58:36.288269+0000","last_clean":"2024-08-15T17:58:36.288269+0000","last_became_active":"2024-08-15T17:58:36.278014+0000","last_became_peered":"2024-08-15T17:58:36.278014+0000","last_unstale":"2024-08-15T17:58:36.288269+0000","last_undegraded":"2024-08-15T17:58:36.288269+0000","last_fullsized":"2024-08-15T17:58:36.288269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:27:54.650775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.274096+0000","last_change":"2024-08-15T17:58:35.278113+0000","last_active":"2024-08-15T17:58:36.274096+0000","last_peered":"2024-08-15T17:58:36.274096+0000","last_clean":"2024-08-15T17:58:36.274096+0000","last_became_active":"2024-08-15T17:58:35.277811+0000","last_became_peered":"2024-08-15T17:58:35.277811+0000","last_unstale":"2024-08-15T17:58:36.274096+0000","last_undegraded":"2024-08-15T17:58:36.274096+0000","last_fullsized":"2024-08-15T17:58:36.274096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:12:40.711380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294826+0000","last_change":"2024-08-15T17:58:36.283999+0000","last_active":"2024-08-15T17:58:36.294826+0000","last_peered":"2024-08-15T17:58:36.294826+0000","last_clean":"2024-08-15T17:58:36.294826+0000","last_became_active":"2024-08-15T17:58:36.283723+0000","last_became_peered":"2024-08-15T17:58:36.283723+0000","last_unstale":"2024-08-15T17:58:36.294826+0000","last_undegraded":"2024-08-15T17:58:36.294826+0000","last_fullsized":"2024-08-15T17:58:36.294826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:23:04.074155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283536+0000","last_change":"2024-08-15T17:58:35.268112+0000","last_active":"2024-08-15T17:58:36.283536+0000","last_peered":"2024-08-15T17:58:36.283536+0000","last_clean":"2024-08-15T17:58:36.283536+0000","last_became_active":"2024-08-15T17:58:35.265298+0000","last_became_peered":"2024-08-15T17:58:35.265298+0000","last_unstale":"2024-08-15T17:58:36.283536+0000","last_undegraded":"2024-08-15T17:58:36.283536+0000","last_fullsized":"2024-08-15T17:58:36.283536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:13.047239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294320+0000","last_change":"2024-08-15T17:58:36.286548+0000","last_active":"2024-08-15T17:58:36.294320+0000","last_peered":"2024-08-15T17:58:36.294320+0000","last_clean":"2024-08-15T17:58:36.294320+0000","last_became_active":"2024-08-15T17:58:36.286287+0000","last_became_peered":"2024-08-15T17:58:36.286287+0000","last_unstale":"2024-08-15T17:58:36.294320+0000","last_undegraded":"2024-08-15T17:58:36.294320+0000","last_fullsized":"2024-08-15T17:58:36.294320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:07:03.605391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.304604+0000","last_change":"2024-08-15T17:58:35.271260+0000","last_active":"2024-08-15T17:58:36.304604+0000","last_peered":"2024-08-15T17:58:36.304604+0000","last_clean":"2024-08-15T17:58:36.304604+0000","last_became_active":"2024-08-15T17:58:35.270816+0000","last_became_peered":"2024-08-15T17:58:35.270816+0000","last_unstale":"2024-08-15T17:58:36.304604+0000","last_undegraded":"2024-08-15T17:58:36.304604+0000","last_fullsized":"2024-08-15T17:58:36.304604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:58:46.815165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.291658+0000","last_change":"2024-08-15T17:58:36.283169+0000","last_active":"2024-08-15T17:58:36.291658+0000","last_peered":"2024-08-15T17:58:36.291658+0000","last_clean":"2024-08-15T17:58:36.291658+0000","last_became_active":"2024-08-15T17:58:36.282882+0000","last_became_peered":"2024-08-15T17:58:36.282882+0000","last_unstale":"2024-08-15T17:58:36.291658+0000","last_undegraded":"2024-08-15T17:58:36.291658+0000","last_fullsized":"2024-08-15T17:58:36.291658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:24:27.743039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.299612+0000","last_change":"2024-08-15T17:58:35.266260+0000","last_active":"2024-08-15T17:58:36.299612+0000","last_peered":"2024-08-15T17:58:36.299612+0000","last_clean":"2024-08-15T17:58:36.299612+0000","last_became_active":"2024-08-15T17:58:35.265941+0000","last_became_peered":"2024-08-15T17:58:35.265941+0000","last_unstale":"2024-08-15T17:58:36.299612+0000","last_undegraded":"2024-08-15T17:58:36.299612+0000","last_fullsized":"2024-08-15T17:58:36.299612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:48:09.608873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.296591+0000","last_change":"2024-08-15T17:58:36.285123+0000","last_active":"2024-08-15T17:58:36.296591+0000","last_peered":"2024-08-15T17:58:36.296591+0000","last_clean":"2024-08-15T17:58:36.296591+0000","last_became_active":"2024-08-15T17:58:36.284782+0000","last_became_peered":"2024-08-15T17:58:36.284782+0000","last_unstale":"2024-08-15T17:58:36.296591+0000","last_undegraded":"2024-08-15T17:58:36.296591+0000","last_fullsized":"2024-08-15T17:58:36.296591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:42:52.458542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301625+0000","last_change":"2024-08-15T17:58:35.273755+0000","last_active":"2024-08-15T17:58:36.301625+0000","last_peered":"2024-08-15T17:58:36.301625+0000","last_clean":"2024-08-15T17:58:36.301625+0000","last_became_active":"2024-08-15T17:58:35.273318+0000","last_became_peered":"2024-08-15T17:58:35.273318+0000","last_unstale":"2024-08-15T17:58:36.301625+0000","last_undegraded":"2024-08-15T17:58:36.301625+0000","last_fullsized":"2024-08-15T17:58:36.301625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:13.047239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.295367+0000","last_change":"2024-08-15T17:58:36.286635+0000","last_active":"2024-08-15T17:58:36.295367+0000","last_peered":"2024-08-15T17:58:36.295367+0000","last_clean":"2024-08-15T17:58:36.295367+0000","last_became_active":"2024-08-15T17:58:36.286062+0000","last_became_peered":"2024-08-15T17:58:36.286062+0000","last_unstale":"2024-08-15T17:58:36.295367+0000","last_undegraded":"2024-08-15T17:58:36.295367+0000","last_fullsized":"2024-08-15T17:58:36.295367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:21:42.887962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.302921+0000","last_change":"2024-08-15T17:58:35.274846+0000","last_active":"2024-08-15T17:58:36.302921+0000","last_peered":"2024-08-15T17:58:36.302921+0000","last_clean":"2024-08-15T17:58:36.302921+0000","last_became_active":"2024-08-15T17:58:35.274543+0000","last_became_peered":"2024-08-15T17:58:35.274543+0000","last_unstale":"2024-08-15T17:58:36.302921+0000","last_undegraded":"2024-08-15T17:58:36.302921+0000","last_fullsized":"2024-08-15T17:58:36.302921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:11:26.241336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288832+0000","last_change":"2024-08-15T17:58:36.285388+0000","last_active":"2024-08-15T17:58:36.288832+0000","last_peered":"2024-08-15T17:58:36.288832+0000","last_clean":"2024-08-15T17:58:36.288832+0000","last_became_active":"2024-08-15T17:58:36.285136+0000","last_became_peered":"2024-08-15T17:58:36.285136+0000","last_unstale":"2024-08-15T17:58:36.288832+0000","last_undegraded":"2024-08-15T17:58:36.288832+0000","last_fullsized":"2024-08-15T17:58:36.288832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:43.158602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.273480+0000","last_change":"2024-08-15T17:58:35.272166+0000","last_active":"2024-08-15T17:58:36.273480+0000","last_peered":"2024-08-15T17:58:36.273480+0000","last_clean":"2024-08-15T17:58:36.273480+0000","last_became_active":"2024-08-15T17:58:35.271841+0000","last_became_peered":"2024-08-15T17:58:35.271841+0000","last_unstale":"2024-08-15T17:58:36.273480+0000","last_undegraded":"2024-08-15T17:58:36.273480+0000","last_fullsized":"2024-08-15T17:58:36.273480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:40:09.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.296052+0000","last_change":"2024-08-15T17:58:36.285039+0000","last_active":"2024-08-15T17:58:36.296052+0000","last_peered":"2024-08-15T17:58:36.296052+0000","last_clean":"2024-08-15T17:58:36.296052+0000","last_became_active":"2024-08-15T17:58:36.284458+0000","last_became_peered":"2024-08-15T17:58:36.284458+0000","last_unstale":"2024-08-15T17:58:36.296052+0000","last_undegraded":"2024-08-15T17:58:36.296052+0000","last_fullsized":"2024-08-15T17:58:36.296052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:37:55.417330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.271012+0000","last_change":"2024-08-15T17:58:24.394014+0000","last_active":"2024-08-15T17:58:36.271012+0000","last_peered":"2024-08-15T17:58:36.271012+0000","last_clean":"2024-08-15T17:58:36.271012+0000","last_became_active":"2024-08-15T17:58:24.391735+0000","last_became_peered":"2024-08-15T17:58:24.391735+0000","last_unstale":"2024-08-15T17:58:36.271012+0000","last_undegraded":"2024-08-15T17:58:36.271012+0000","last_fullsized":"2024-08-15T17:58:36.271012+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_clean_scrub_stamp":"2024-08-15T17:58:22.996335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:31.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.273936+0000","last_change":"2024-08-15T17:58:35.269765+0000","last_active":"2024-08-15T17:58:36.273936+0000","last_peered":"2024-08-15T17:58:36.273936+0000","last_clean":"2024-08-15T17:58:36.273936+0000","last_became_active":"2024-08-15T17:58:35.269249+0000","last_became_peered":"2024-08-15T17:58:35.269249+0000","last_unstale":"2024-08-15T17:58:36.273936+0000","last_undegraded":"2024-08-15T17:58:36.273936+0000","last_fullsized":"2024-08-15T17:58:36.273936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:27:34.522329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.292584+0000","last_change":"2024-08-15T17:58:36.286437+0000","last_active":"2024-08-15T17:58:36.292584+0000","last_peered":"2024-08-15T17:58:36.292584+0000","last_clean":"2024-08-15T17:58:36.292584+0000","last_became_active":"2024-08-15T17:58:36.285909+0000","last_became_peered":"2024-08-15T17:58:36.285909+0000","last_unstale":"2024-08-15T17:58:36.292584+0000","last_undegraded":"2024-08-15T17:58:36.292584+0000","last_fullsized":"2024-08-15T17:58:36.292584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:47:17.486762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.270935+0000","last_change":"2024-08-15T17:58:35.275547+0000","last_active":"2024-08-15T17:58:36.270935+0000","last_peered":"2024-08-15T17:58:36.270935+0000","last_clean":"2024-08-15T17:58:36.270935+0000","last_became_active":"2024-08-15T17:58:35.275253+0000","last_became_peered":"2024-08-15T17:58:35.275253+0000","last_unstale":"2024-08-15T17:58:36.270935+0000","last_undegraded":"2024-08-15T17:58:36.270935+0000","last_fullsized":"2024-08-15T17:58:36.270935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:44.174826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288317+0000","last_change":"2024-08-15T17:58:36.277792+0000","last_active":"2024-08-15T17:58:36.288317+0000","last_peered":"2024-08-15T17:58:36.288317+0000","last_clean":"2024-08-15T17:58:36.288317+0000","last_became_active":"2024-08-15T17:58:36.276715+0000","last_became_peered":"2024-08-15T17:58:36.276715+0000","last_unstale":"2024-08-15T17:58:36.288317+0000","last_undegraded":"2024-08-15T17:58:36.288317+0000","last_fullsized":"2024-08-15T17:58:36.288317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:59:27.743235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.275699+0000","last_change":"2024-08-15T17:58:35.280834+0000","last_active":"2024-08-15T17:58:36.275699+0000","last_peered":"2024-08-15T17:58:36.275699+0000","last_clean":"2024-08-15T17:58:36.275699+0000","last_became_active":"2024-08-15T17:58:35.280524+0000","last_became_peered":"2024-08-15T17:58:35.280524+0000","last_unstale":"2024-08-15T17:58:36.275699+0000","last_undegraded":"2024-08-15T17:58:36.275699+0000","last_fullsized":"2024-08-15T17:58:36.275699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:00:50.031787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289140+0000","last_change":"2024-08-15T17:58:36.283772+0000","last_active":"2024-08-15T17:58:36.289140+0000","last_peered":"2024-08-15T17:58:36.289140+0000","last_clean":"2024-08-15T17:58:36.289140+0000","last_became_active":"2024-08-15T17:58:36.283482+0000","last_became_peered":"2024-08-15T17:58:36.283482+0000","last_unstale":"2024-08-15T17:58:36.289140+0000","last_undegraded":"2024-08-15T17:58:36.289140+0000","last_fullsized":"2024-08-15T17:58:36.289140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:33.556488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.274203+0000","last_change":"2024-08-15T17:58:35.270088+0000","last_active":"2024-08-15T17:58:36.274203+0000","last_peered":"2024-08-15T17:58:36.274203+0000","last_clean":"2024-08-15T17:58:36.274203+0000","last_became_active":"2024-08-15T17:58:35.268586+0000","last_became_peered":"2024-08-15T17:58:35.268586+0000","last_unstale":"2024-08-15T17:58:36.274203+0000","last_undegraded":"2024-08-15T17:58:36.274203+0000","last_fullsized":"2024-08-15T17:58:36.274203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:17:09.213636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285313+0000","last_change":"2024-08-15T17:58:36.277552+0000","last_active":"2024-08-15T17:58:36.285313+0000","last_peered":"2024-08-15T17:58:36.285313+0000","last_clean":"2024-08-15T17:58:36.285313+0000","last_became_active":"2024-08-15T17:58:36.277173+0000","last_became_peered":"2024-08-15T17:58:36.277173+0000","last_unstale":"2024-08-15T17:58:36.285313+0000","last_undegraded":"2024-08-15T17:58:36.285313+0000","last_fullsized":"2024-08-15T17:58:36.285313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:18.599017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.272956+0000","last_change":"2024-08-15T17:58:35.270974+0000","last_active":"2024-08-15T17:58:36.272956+0000","last_peered":"2024-08-15T17:58:36.272956+0000","last_clean":"2024-08-15T17:58:36.272956+0000","last_became_active":"2024-08-15T17:58:35.270645+0000","last_became_peered":"2024-08-15T17:58:35.270645+0000","last_unstale":"2024-08-15T17:58:36.272956+0000","last_undegraded":"2024-08-15T17:58:36.272956+0000","last_fullsized":"2024-08-15T17:58:36.272956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:44.174826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288251+0000","last_change":"2024-08-15T17:58:36.282072+0000","last_active":"2024-08-15T17:58:36.288251+0000","last_peered":"2024-08-15T17:58:36.288251+0000","last_clean":"2024-08-15T17:58:36.288251+0000","last_became_active":"2024-08-15T17:58:36.281850+0000","last_became_peered":"2024-08-15T17:58:36.281850+0000","last_unstale":"2024-08-15T17:58:36.288251+0000","last_undegraded":"2024-08-15T17:58:36.288251+0000","last_fullsized":"2024-08-15T17:58:36.288251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:54:56.996071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.277028+0000","last_change":"2024-08-15T17:58:35.275830+0000","last_active":"2024-08-15T17:58:36.277028+0000","last_peered":"2024-08-15T17:58:36.277028+0000","last_clean":"2024-08-15T17:58:36.277028+0000","last_became_active":"2024-08-15T17:58:35.275532+0000","last_became_peered":"2024-08-15T17:58:35.275532+0000","last_unstale":"2024-08-15T17:58:36.277028+0000","last_undegraded":"2024-08-15T17:58:36.277028+0000","last_fullsized":"2024-08-15T17:58:36.277028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:38.845281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289260+0000","last_change":"2024-08-15T17:58:36.284264+0000","last_active":"2024-08-15T17:58:36.289260+0000","last_peered":"2024-08-15T17:58:36.289260+0000","last_clean":"2024-08-15T17:58:36.289260+0000","last_became_active":"2024-08-15T17:58:36.283964+0000","last_became_peered":"2024-08-15T17:58:36.283964+0000","last_unstale":"2024-08-15T17:58:36.289260+0000","last_undegraded":"2024-08-15T17:58:36.289260+0000","last_fullsized":"2024-08-15T17:58:36.289260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:07:03.605391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-15T17:58:37.297038+0000","last_change":"2024-08-15T17:58:35.274546+0000","last_active":"2024-08-15T17:58:37.297038+0000","last_peered":"2024-08-15T17:58:37.297038+0000","last_clean":"2024-08-15T17:58:37.297038+0000","last_became_active":"2024-08-15T17:58:35.274232+0000","last_became_peered":"2024-08-15T17:58:35.274232+0000","last_unstale":"2024-08-15T17:58:37.297038+0000","last_undegraded":"2024-08-15T17:58:37.297038+0000","last_fullsized":"2024-08-15T17:58:37.297038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:17.590038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293817+0000","last_change":"2024-08-15T17:58:36.290821+0000","last_active":"2024-08-15T17:58:36.293817+0000","last_peered":"2024-08-15T17:58:36.293817+0000","last_clean":"2024-08-15T17:58:36.293817+0000","last_became_active":"2024-08-15T17:58:36.290495+0000","last_became_peered":"2024-08-15T17:58:36.290495+0000","last_unstale":"2024-08-15T17:58:36.293817+0000","last_undegraded":"2024-08-15T17:58:36.293817+0000","last_fullsized":"2024-08-15T17:58:36.293817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:43.158602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298969+0000","last_change":"2024-08-15T17:58:35.274034+0000","last_active":"2024-08-15T17:58:36.298969+0000","last_peered":"2024-08-15T17:58:36.298969+0000","last_clean":"2024-08-15T17:58:36.298969+0000","last_became_active":"2024-08-15T17:58:35.273028+0000","last_became_peered":"2024-08-15T17:58:35.273028+0000","last_unstale":"2024-08-15T17:58:36.298969+0000","last_undegraded":"2024-08-15T17:58:36.298969+0000","last_fullsized":"2024-08-15T17:58:36.298969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:26:33.980860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289390+0000","last_change":"2024-08-15T17:58:36.284750+0000","last_active":"2024-08-15T17:58:36.289390+0000","last_peered":"2024-08-15T17:58:36.289390+0000","last_clean":"2024-08-15T17:58:36.289390+0000","last_became_active":"2024-08-15T17:58:36.284480+0000","last_became_peered":"2024-08-15T17:58:36.284480+0000","last_unstale":"2024-08-15T17:58:36.289390+0000","last_undegraded":"2024-08-15T17:58:36.289390+0000","last_fullsized":"2024-08-15T17:58:36.289390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:55.518837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.270849+0000","last_change":"2024-08-15T17:58:35.272396+0000","last_active":"2024-08-15T17:58:36.270849+0000","last_peered":"2024-08-15T17:58:36.270849+0000","last_clean":"2024-08-15T17:58:36.270849+0000","last_became_active":"2024-08-15T17:58:35.272141+0000","last_became_peered":"2024-08-15T17:58:35.272141+0000","last_unstale":"2024-08-15T17:58:36.270849+0000","last_undegraded":"2024-08-15T17:58:36.270849+0000","last_fullsized":"2024-08-15T17:58:36.270849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:17.590038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286071+0000","last_change":"2024-08-15T17:58:36.278104+0000","last_active":"2024-08-15T17:58:36.286071+0000","last_peered":"2024-08-15T17:58:36.286071+0000","last_clean":"2024-08-15T17:58:36.286071+0000","last_became_active":"2024-08-15T17:58:36.277844+0000","last_became_peered":"2024-08-15T17:58:36.277844+0000","last_unstale":"2024-08-15T17:58:36.286071+0000","last_undegraded":"2024-08-15T17:58:36.286071+0000","last_fullsized":"2024-08-15T17:58:36.286071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:26:34.989839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.275661+0000","last_change":"2024-08-15T17:58:35.277768+0000","last_active":"2024-08-15T17:58:36.275661+0000","last_peered":"2024-08-15T17:58:36.275661+0000","last_clean":"2024-08-15T17:58:36.275661+0000","last_became_active":"2024-08-15T17:58:35.277217+0000","last_became_peered":"2024-08-15T17:58:35.277217+0000","last_unstale":"2024-08-15T17:58:36.275661+0000","last_undegraded":"2024-08-15T17:58:36.275661+0000","last_fullsized":"2024-08-15T17:58:36.275661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:13.047239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293613+0000","last_change":"2024-08-15T17:58:36.289433+0000","last_active":"2024-08-15T17:58:36.293613+0000","last_peered":"2024-08-15T17:58:36.293613+0000","last_clean":"2024-08-15T17:58:36.293613+0000","last_became_active":"2024-08-15T17:58:36.288625+0000","last_became_peered":"2024-08-15T17:58:36.288625+0000","last_unstale":"2024-08-15T17:58:36.293613+0000","last_undegraded":"2024-08-15T17:58:36.293613+0000","last_fullsized":"2024-08-15T17:58:36.293613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:47:03.102185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.273570+0000","last_change":"2024-08-15T17:58:35.276117+0000","last_active":"2024-08-15T17:58:36.273570+0000","last_peered":"2024-08-15T17:58:36.273570+0000","last_clean":"2024-08-15T17:58:36.273570+0000","last_became_active":"2024-08-15T17:58:35.275707+0000","last_became_peered":"2024-08-15T17:58:35.275707+0000","last_unstale":"2024-08-15T17:58:36.273570+0000","last_undegraded":"2024-08-15T17:58:36.273570+0000","last_fullsized":"2024-08-15T17:58:36.273570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:40:09.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293636+0000","last_change":"2024-08-15T17:58:36.290279+0000","last_active":"2024-08-15T17:58:36.293636+0000","last_peered":"2024-08-15T17:58:36.293636+0000","last_clean":"2024-08-15T17:58:36.293636+0000","last_became_active":"2024-08-15T17:58:36.289946+0000","last_became_peered":"2024-08-15T17:58:36.289946+0000","last_unstale":"2024-08-15T17:58:36.293636+0000","last_undegraded":"2024-08-15T17:58:36.293636+0000","last_fullsized":"2024-08-15T17:58:36.293636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298629+0000","last_change":"2024-08-15T17:58:35.274655+0000","last_active":"2024-08-15T17:58:36.298629+0000","last_peered":"2024-08-15T17:58:36.298629+0000","last_clean":"2024-08-15T17:58:36.298629+0000","last_became_active":"2024-08-15T17:58:35.273860+0000","last_became_peered":"2024-08-15T17:58:35.273860+0000","last_unstale":"2024-08-15T17:58:36.298629+0000","last_undegraded":"2024-08-15T17:58:36.298629+0000","last_fullsized":"2024-08-15T17:58:36.298629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:32:57.548918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294347+0000","last_change":"2024-08-15T17:58:36.285943+0000","last_active":"2024-08-15T17:58:36.294347+0000","last_peered":"2024-08-15T17:58:36.294347+0000","last_clean":"2024-08-15T17:58:36.294347+0000","last_became_active":"2024-08-15T17:58:36.283605+0000","last_became_peered":"2024-08-15T17:58:36.283605+0000","last_unstale":"2024-08-15T17:58:36.294347+0000","last_undegraded":"2024-08-15T17:58:36.294347+0000","last_fullsized":"2024-08-15T17:58:36.294347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:07:28.572500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.282145+0000","last_change":"2024-08-15T17:58:36.271574+0000","last_active":"2024-08-15T17:58:36.282145+0000","last_peered":"2024-08-15T17:58:36.282145+0000","last_clean":"2024-08-15T17:58:36.282145+0000","last_became_active":"2024-08-15T17:58:36.271259+0000","last_became_peered":"2024-08-15T17:58:36.271259+0000","last_unstale":"2024-08-15T17:58:36.282145+0000","last_undegraded":"2024-08-15T17:58:36.282145+0000","last_fullsized":"2024-08-15T17:58:36.282145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298518+0000","last_change":"2024-08-15T17:58:35.277718+0000","last_active":"2024-08-15T17:58:36.298518+0000","last_peered":"2024-08-15T17:58:36.298518+0000","last_clean":"2024-08-15T17:58:36.298518+0000","last_became_active":"2024-08-15T17:58:35.277447+0000","last_became_peered":"2024-08-15T17:58:35.277447+0000","last_unstale":"2024-08-15T17:58:36.298518+0000","last_undegraded":"2024-08-15T17:58:36.298518+0000","last_fullsized":"2024-08-15T17:58:36.298518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:23:08.234362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288214+0000","last_change":"2024-08-15T17:58:36.281039+0000","last_active":"2024-08-15T17:58:36.288214+0000","last_peered":"2024-08-15T17:58:36.288214+0000","last_clean":"2024-08-15T17:58:36.288214+0000","last_became_active":"2024-08-15T17:58:36.280780+0000","last_became_peered":"2024-08-15T17:58:36.280780+0000","last_unstale":"2024-08-15T17:58:36.288214+0000","last_undegraded":"2024-08-15T17:58:36.288214+0000","last_fullsized":"2024-08-15T17:58:36.288214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:47:03.102185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298615+0000","last_change":"2024-08-15T17:58:35.276674+0000","last_active":"2024-08-15T17:58:36.298615+0000","last_peered":"2024-08-15T17:58:36.298615+0000","last_clean":"2024-08-15T17:58:36.298615+0000","last_became_active":"2024-08-15T17:58:35.276363+0000","last_became_peered":"2024-08-15T17:58:35.276363+0000","last_unstale":"2024-08-15T17:58:36.298615+0000","last_undegraded":"2024-08-15T17:58:36.298615+0000","last_fullsized":"2024-08-15T17:58:36.298615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:20.838352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288357+0000","last_change":"2024-08-15T17:58:36.282308+0000","last_active":"2024-08-15T17:58:36.288357+0000","last_peered":"2024-08-15T17:58:36.288357+0000","last_clean":"2024-08-15T17:58:36.288357+0000","last_became_active":"2024-08-15T17:58:36.281986+0000","last_became_peered":"2024-08-15T17:58:36.281986+0000","last_unstale":"2024-08-15T17:58:36.288357+0000","last_undegraded":"2024-08-15T17:58:36.288357+0000","last_fullsized":"2024-08-15T17:58:36.288357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:48:28.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.306258+0000","last_change":"2024-08-15T17:58:35.274614+0000","last_active":"2024-08-15T17:58:36.306258+0000","last_peered":"2024-08-15T17:58:36.306258+0000","last_clean":"2024-08-15T17:58:36.306258+0000","last_became_active":"2024-08-15T17:58:35.274185+0000","last_became_peered":"2024-08-15T17:58:35.274185+0000","last_unstale":"2024-08-15T17:58:36.306258+0000","last_undegraded":"2024-08-15T17:58:36.306258+0000","last_fullsized":"2024-08-15T17:58:36.306258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:41.642352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293725+0000","last_change":"2024-08-15T17:58:36.290572+0000","last_active":"2024-08-15T17:58:36.293725+0000","last_peered":"2024-08-15T17:58:36.293725+0000","last_clean":"2024-08-15T17:58:36.293725+0000","last_became_active":"2024-08-15T17:58:36.290269+0000","last_became_peered":"2024-08-15T17:58:36.290269+0000","last_unstale":"2024-08-15T17:58:36.293725+0000","last_undegraded":"2024-08-15T17:58:36.293725+0000","last_fullsized":"2024-08-15T17:58:36.293725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:56:49.069407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301957+0000","last_change":"2024-08-15T17:58:35.276392+0000","last_active":"2024-08-15T17:58:36.301957+0000","last_peered":"2024-08-15T17:58:36.301957+0000","last_clean":"2024-08-15T17:58:36.301957+0000","last_became_active":"2024-08-15T17:58:35.276085+0000","last_became_peered":"2024-08-15T17:58:35.276085+0000","last_unstale":"2024-08-15T17:58:36.301957+0000","last_undegraded":"2024-08-15T17:58:36.301957+0000","last_fullsized":"2024-08-15T17:58:36.301957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:10:36.097809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":806912,"data_stored":663739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]}]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":74353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":72927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9419999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":212992,"data_stored":72891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":74353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301724,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070116,"statfs":{"total":96636764160,"available":96327798784,"internally_reserved":0,"allocated":208896,"data_stored":72801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301728,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070112,"statfs":{"total":96636764160,"available":96327794688,"internally_reserved":0,"allocated":217088,"data_stored":72961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298160,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073680,"statfs":{"total":96636764160,"available":96331448320,"internally_reserved":0,"allocated":815104,"data_stored":663795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:58:43.951 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:58:44.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e53302700 1 Processor -- start 2024-08-15T17:58:44.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e53302700 1 -- start start 2024-08-15T17:58:44.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e53302700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1e4c0af340 0x7f1e4c067840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e53302700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c0cc8c0 con 0x7f1e4c067d80 2024-08-15T17:58:44.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e53302700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c0cbca0 con 0x7f1e4c156d80 2024-08-15T17:58:44.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e53302700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4c068200 con 0x7f1e4c0af340 2024-08-15T17:58:44.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e52b01700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1e4c067d80 0x7f1e4c157150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40222/0 (socket says 172.21.15.38:40222) 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e52b01700 1 -- 172.21.15.38:0/2941061902 learned_addr learned my addr 172.21.15.38:0/2941061902 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555553699 0 0) 0x7f1e4c0cc8c0 con 0x7f1e4c067d80 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e38003570 con 0x7f1e4c067d80 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e51aff700 1 --2- 172.21.15.38:0/2941061902 >> v2:172.21.15.104:3300/0 conn(0x7f1e4c0af340 0x7f1e4c067840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2175828142 0 0) 0x7f1e38003570 con 0x7f1e4c067d80 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 >> v2:172.21.15.104:3300/0 conn(0x7f1e4c0af340 msgr2=0x7f1e4c067840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 --2- 172.21.15.38:0/2941061902 >> v2:172.21.15.104:3300/0 conn(0x7f1e4c0af340 0x7f1e4c067840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 >> v1:172.21.15.104:6789/0 conn(0x7f1e4c156d80 legacy=0x7f1e4c0aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c068450 con 0x7f1e4c067d80 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f1e440017f0 con 0x7f1e4c067d80 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 >> v1:172.21.15.38:6789/0 conn(0x7f1e4c067d80 legacy=0x7f1e4c157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 --2- 172.21.15.38:0/2941061902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e38003a60 0x7f1e38003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 --2- 172.21.15.38:0/2941061902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e38004810 0x7f1e38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 --2- 172.21.15.38:0/2941061902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e380052e0 0x7f1e3800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c068200 con 0x7f1e380052e0 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3800c190 con 0x7f1e38004810 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3800c350 con 0x7f1e38003a60 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e51aff700 1 --2- 172.21.15.38:0/2941061902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e38004810 0x7f1e38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e52b01700 1 --2- 172.21.15.38:0/2941061902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e380052e0 0x7f1e3800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e52300700 1 --2- 172.21.15.38:0/2941061902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e38003a60 0x7f1e38003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e51aff700 1 -- 172.21.15.38:0/2941061902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e38003a60 msgr2=0x7f1e38003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e51aff700 1 --2- 172.21.15.38:0/2941061902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e38003a60 0x7f1e38003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e51aff700 1 -- 172.21.15.38:0/2941061902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e380052e0 msgr2=0x7f1e3800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e51aff700 1 --2- 172.21.15.38:0/2941061902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e380052e0 0x7f1e3800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e51aff700 1 -- 172.21.15.38:0/2941061902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e38003570 con 0x7f1e38004810 2024-08-15T17:58:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e52b01700 1 -- 172.21.15.38:0/2941061902 reap_dead start 2024-08-15T17:58:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e51aff700 1 --2- 172.21.15.38:0/2941061902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e38004810 0x7f1e38004c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00bb60 tx=0x7f1e3c008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28278902d68a8eb5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00d5d0 con 0x7f1e38004810 2024-08-15T17:58:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3c00d730 con 0x7f1e38004810 2024-08-15T17:58:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.054+0000 7f1e512fe700 1 -- 172.21.15.38:0/2941061902 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00daf0 con 0x7f1e38004810 2024-08-15T17:58:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 -- 172.21.15.38:0/2941061902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e38004810 msgr2=0x7f1e38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 --2- 172.21.15.38:0/2941061902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e38004810 0x7f1e38004c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00bb60 tx=0x7f1e3c008040 comp rx=0 tx=0).stop 2024-08-15T17:58:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 -- 172.21.15.38:0/2941061902 shutdown_connections 2024-08-15T17:58:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 --2- 172.21.15.38:0/2941061902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e38004810 0x7f1e38004c80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 -- 172.21.15.38:0/2941061902 >> 172.21.15.38:0/2941061902 conn(0x7f1e4c06e070 msgr2=0x7f1e4c0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 -- 172.21.15.38:0/2941061902 shutdown_connections 2024-08-15T17:58:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 -- 172.21.15.38:0/2941061902 wait complete. 2024-08-15T17:58:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 Processor -- start 2024-08-15T17:58:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 -- start start 2024-08-15T17:58:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e38004810 0x7f1e4c14bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e4c14f720 0x7f1e4c159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e4c14fb90 0x7f1e4c15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c15a850 con 0x7f1e4c14f720 2024-08-15T17:58:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c15a640 con 0x7f1e38004810 2024-08-15T17:58:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.058+0000 7f1e53302700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c154870 con 0x7f1e4c14fb90 2024-08-15T17:58:44.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e51aff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e4c14f720 0x7f1e4c159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e51aff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e4c14f720 0x7f1e4c159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53984/0 (socket says 172.21.15.38:53984) 2024-08-15T17:58:44.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e51aff700 1 -- 172.21.15.38:0/3317940826 learned_addr learned my addr 172.21.15.38:0/3317940826 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:44.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e52b01700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e4c14fb90 0x7f1e4c15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e52300700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e38004810 0x7f1e4c14bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e51aff700 1 -- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e4c14fb90 msgr2=0x7f1e4c15a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e51aff700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e4c14fb90 0x7f1e4c15a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e51aff700 1 -- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e38004810 msgr2=0x7f1e4c14bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e51aff700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e38004810 0x7f1e4c14bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e51aff700 1 -- 172.21.15.38:0/3317940826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c154b50 con 0x7f1e4c14f720 2024-08-15T17:58:44.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e51aff700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e4c14f720 0x7f1e4c159bc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800c320 tx=0x7f1e3c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e427fc700 1 -- 172.21.15.38:0/3317940826 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c014740 con 0x7f1e4c14f720 2024-08-15T17:58:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e427fc700 1 -- 172.21.15.38:0/3317940826 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3c0148c0 con 0x7f1e4c14f720 2024-08-15T17:58:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e53302700 1 -- 172.21.15.38:0/3317940826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c154e30 con 0x7f1e4c14f720 2024-08-15T17:58:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e427fc700 1 -- 172.21.15.38:0/3317940826 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c014c80 con 0x7f1e4c14f720 2024-08-15T17:58:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e53302700 1 -- 172.21.15.38:0/3317940826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4c14c290 con 0x7f1e4c14f720 2024-08-15T17:58:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.062+0000 7f1e53302700 1 -- 172.21.15.38:0/3317940826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4c0638f0 con 0x7f1e4c14f720 2024-08-15T17:58:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.066+0000 7f1e427fc700 1 -- 172.21.15.38:0/3317940826 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1e3c01d610 con 0x7f1e4c14f720 2024-08-15T17:58:44.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.066+0000 7f1e427fc700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1e30060890 0x7f1e30062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.066+0000 7f1e427fc700 1 -- 172.21.15.38:0/3317940826 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e3c07df10 con 0x7f1e4c14f720 2024-08-15T17:58:44.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.066+0000 7f1e52300700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1e30060890 0x7f1e30062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.066+0000 7f1e52300700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1e30060890 0x7f1e30062d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e48000c00 tx=0x7f1e48007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.074+0000 7f1e427fc700 1 -- 172.21.15.38:0/3317940826 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1e3c04e420 con 0x7f1e4c14f720 2024-08-15T17:58:44.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.230+0000 7f1e53302700 1 -- 172.21.15.38:0/3317940826 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e4c1549e0 con 0x7f1e30060890 2024-08-15T17:58:44.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.246+0000 7f1e427fc700 1 -- 172.21.15.38:0/3317940826 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264387 (secure 0 0 0) 0x7f1e4c1549e0 con 0x7f1e30060890 2024-08-15T17:58:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:44.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 -- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1e30060890 msgr2=0x7f1e30062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1e30060890 0x7f1e30062d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e48000c00 tx=0x7f1e48007d30 comp rx=0 tx=0).stop 2024-08-15T17:58:44.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 -- 172.21.15.38:0/3317940826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e4c14f720 msgr2=0x7f1e4c159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e4c14f720 0x7f1e4c159bc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800c320 tx=0x7f1e3c01b000 comp rx=0 tx=0).stop 2024-08-15T17:58:44.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 -- 172.21.15.38:0/3317940826 shutdown_connections 2024-08-15T17:58:44.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1e30060890 0x7f1e30062d40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e4c14fb90 0x7f1e4c15a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e4c14f720 0x7f1e4c159bc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 --2- 172.21.15.38:0/3317940826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e38004810 0x7f1e4c14bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 -- 172.21.15.38:0/3317940826 >> 172.21.15.38:0/3317940826 conn(0x7f1e4c06e070 msgr2=0x7f1e4c0c9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:44.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 -- 172.21.15.38:0/3317940826 shutdown_connections 2024-08-15T17:58:44.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.250+0000 7f1e53302700 1 -- 172.21.15.38:0/3317940826 wait complete. 2024-08-15T17:58:44.257 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-08-15T17:58:44.267 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-08-15T17:58:44.170592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2390044,"kb_used_data":2856,"kb_used_omap":13,"kb_used_meta":2387058,"kb_avail":752584676,"statfs":{"total":773094113280,"available":770646708224,"internally_reserved":0,"allocated":2924544,"data_stored":1767820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2444348096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003068"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.295385+0000","last_change":"2024-08-15T17:58:36.292335+0000","last_active":"2024-08-15T17:58:36.295385+0000","last_peered":"2024-08-15T17:58:36.295385+0000","last_clean":"2024-08-15T17:58:36.295385+0000","last_became_active":"2024-08-15T17:58:36.292033+0000","last_became_peered":"2024-08-15T17:58:36.292033+0000","last_unstale":"2024-08-15T17:58:36.295385+0000","last_undegraded":"2024-08-15T17:58:36.295385+0000","last_fullsized":"2024-08-15T17:58:36.295385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:59:24.721036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289389+0000","last_change":"2024-08-15T17:58:36.286540+0000","last_active":"2024-08-15T17:58:36.289389+0000","last_peered":"2024-08-15T17:58:36.289389+0000","last_clean":"2024-08-15T17:58:36.289389+0000","last_became_active":"2024-08-15T17:58:36.285657+0000","last_became_peered":"2024-08-15T17:58:36.285657+0000","last_unstale":"2024-08-15T17:58:36.289389+0000","last_undegraded":"2024-08-15T17:58:36.289389+0000","last_fullsized":"2024-08-15T17:58:36.289389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:50:39.313237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288210+0000","last_change":"2024-08-15T17:58:36.279817+0000","last_active":"2024-08-15T17:58:36.288210+0000","last_peered":"2024-08-15T17:58:36.288210+0000","last_clean":"2024-08-15T17:58:36.288210+0000","last_became_active":"2024-08-15T17:58:36.279494+0000","last_became_peered":"2024-08-15T17:58:36.279494+0000","last_unstale":"2024-08-15T17:58:36.288210+0000","last_undegraded":"2024-08-15T17:58:36.288210+0000","last_fullsized":"2024-08-15T17:58:36.288210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:48:29.832597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288855+0000","last_change":"2024-08-15T17:58:36.285873+0000","last_active":"2024-08-15T17:58:36.288855+0000","last_peered":"2024-08-15T17:58:36.288855+0000","last_clean":"2024-08-15T17:58:36.288855+0000","last_became_active":"2024-08-15T17:58:36.285630+0000","last_became_peered":"2024-08-15T17:58:36.285630+0000","last_unstale":"2024-08-15T17:58:36.288855+0000","last_undegraded":"2024-08-15T17:58:36.288855+0000","last_fullsized":"2024-08-15T17:58:36.288855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:30:51.740686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.291564+0000","last_change":"2024-08-15T17:58:36.281869+0000","last_active":"2024-08-15T17:58:36.291564+0000","last_peered":"2024-08-15T17:58:36.291564+0000","last_clean":"2024-08-15T17:58:36.291564+0000","last_became_active":"2024-08-15T17:58:36.281550+0000","last_became_peered":"2024-08-15T17:58:36.281550+0000","last_unstale":"2024-08-15T17:58:36.291564+0000","last_undegraded":"2024-08-15T17:58:36.291564+0000","last_fullsized":"2024-08-15T17:58:36.291564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:33:16.117022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288868+0000","last_change":"2024-08-15T17:58:36.286578+0000","last_active":"2024-08-15T17:58:36.288868+0000","last_peered":"2024-08-15T17:58:36.288868+0000","last_clean":"2024-08-15T17:58:36.288868+0000","last_became_active":"2024-08-15T17:58:36.286298+0000","last_became_peered":"2024-08-15T17:58:36.286298+0000","last_unstale":"2024-08-15T17:58:36.288868+0000","last_undegraded":"2024-08-15T17:58:36.288868+0000","last_fullsized":"2024-08-15T17:58:36.288868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:47:17.486762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.295000+0000","last_change":"2024-08-15T17:58:36.286906+0000","last_active":"2024-08-15T17:58:36.295000+0000","last_peered":"2024-08-15T17:58:36.295000+0000","last_clean":"2024-08-15T17:58:36.295000+0000","last_became_active":"2024-08-15T17:58:36.286620+0000","last_became_peered":"2024-08-15T17:58:36.286620+0000","last_unstale":"2024-08-15T17:58:36.295000+0000","last_undegraded":"2024-08-15T17:58:36.295000+0000","last_fullsized":"2024-08-15T17:58:36.295000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:35:39.618955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.287670+0000","last_change":"2024-08-15T17:58:36.285830+0000","last_active":"2024-08-15T17:58:36.287670+0000","last_peered":"2024-08-15T17:58:36.287670+0000","last_clean":"2024-08-15T17:58:36.287670+0000","last_became_active":"2024-08-15T17:58:36.285422+0000","last_became_peered":"2024-08-15T17:58:36.285422+0000","last_unstale":"2024-08-15T17:58:36.287670+0000","last_undegraded":"2024-08-15T17:58:36.287670+0000","last_fullsized":"2024-08-15T17:58:36.287670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:15:59.759500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.295346+0000","last_change":"2024-08-15T17:58:36.286462+0000","last_active":"2024-08-15T17:58:36.295346+0000","last_peered":"2024-08-15T17:58:36.295346+0000","last_clean":"2024-08-15T17:58:36.295346+0000","last_became_active":"2024-08-15T17:58:36.285671+0000","last_became_peered":"2024-08-15T17:58:36.285671+0000","last_unstale":"2024-08-15T17:58:36.295346+0000","last_undegraded":"2024-08-15T17:58:36.295346+0000","last_fullsized":"2024-08-15T17:58:36.295346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:22:51.492443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294455+0000","last_change":"2024-08-15T17:58:36.289845+0000","last_active":"2024-08-15T17:58:36.294455+0000","last_peered":"2024-08-15T17:58:36.294455+0000","last_clean":"2024-08-15T17:58:36.294455+0000","last_became_active":"2024-08-15T17:58:36.286963+0000","last_became_peered":"2024-08-15T17:58:36.286963+0000","last_unstale":"2024-08-15T17:58:36.294455+0000","last_undegraded":"2024-08-15T17:58:36.294455+0000","last_fullsized":"2024-08-15T17:58:36.294455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:49:19.919158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.295045+0000","last_change":"2024-08-15T17:58:36.292046+0000","last_active":"2024-08-15T17:58:36.295045+0000","last_peered":"2024-08-15T17:58:36.295045+0000","last_clean":"2024-08-15T17:58:36.295045+0000","last_became_active":"2024-08-15T17:58:36.291772+0000","last_became_peered":"2024-08-15T17:58:36.291772+0000","last_unstale":"2024-08-15T17:58:36.295045+0000","last_undegraded":"2024-08-15T17:58:36.295045+0000","last_fullsized":"2024-08-15T17:58:36.295045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:11:45.354221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.291455+0000","last_change":"2024-08-15T17:58:36.288242+0000","last_active":"2024-08-15T17:58:36.291455+0000","last_peered":"2024-08-15T17:58:36.291455+0000","last_clean":"2024-08-15T17:58:36.291455+0000","last_became_active":"2024-08-15T17:58:36.287991+0000","last_became_peered":"2024-08-15T17:58:36.287991+0000","last_unstale":"2024-08-15T17:58:36.291455+0000","last_undegraded":"2024-08-15T17:58:36.291455+0000","last_fullsized":"2024-08-15T17:58:36.291455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:11:27.250315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.292602+0000","last_change":"2024-08-15T17:58:36.286478+0000","last_active":"2024-08-15T17:58:36.292602+0000","last_peered":"2024-08-15T17:58:36.292602+0000","last_clean":"2024-08-15T17:58:36.292602+0000","last_became_active":"2024-08-15T17:58:36.285967+0000","last_became_peered":"2024-08-15T17:58:36.285967+0000","last_unstale":"2024-08-15T17:58:36.292602+0000","last_undegraded":"2024-08-15T17:58:36.292602+0000","last_fullsized":"2024-08-15T17:58:36.292602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288295+0000","last_change":"2024-08-15T17:58:36.279946+0000","last_active":"2024-08-15T17:58:36.288295+0000","last_peered":"2024-08-15T17:58:36.288295+0000","last_clean":"2024-08-15T17:58:36.288295+0000","last_became_active":"2024-08-15T17:58:36.279648+0000","last_became_peered":"2024-08-15T17:58:36.279648+0000","last_unstale":"2024-08-15T17:58:36.288295+0000","last_undegraded":"2024-08-15T17:58:36.288295+0000","last_fullsized":"2024-08-15T17:58:36.288295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:34:14.015831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289440+0000","last_change":"2024-08-15T17:58:36.286890+0000","last_active":"2024-08-15T17:58:36.289440+0000","last_peered":"2024-08-15T17:58:36.289440+0000","last_clean":"2024-08-15T17:58:36.289440+0000","last_became_active":"2024-08-15T17:58:36.286332+0000","last_became_peered":"2024-08-15T17:58:36.286332+0000","last_unstale":"2024-08-15T17:58:36.289440+0000","last_undegraded":"2024-08-15T17:58:36.289440+0000","last_fullsized":"2024-08-15T17:58:36.289440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:07:28.572500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293942+0000","last_change":"2024-08-15T17:58:36.287994+0000","last_active":"2024-08-15T17:58:36.293942+0000","last_peered":"2024-08-15T17:58:36.293942+0000","last_clean":"2024-08-15T17:58:36.293942+0000","last_became_active":"2024-08-15T17:58:36.286645+0000","last_became_peered":"2024-08-15T17:58:36.286645+0000","last_unstale":"2024-08-15T17:58:36.293942+0000","last_undegraded":"2024-08-15T17:58:36.293942+0000","last_fullsized":"2024-08-15T17:58:36.293942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285495+0000","last_change":"2024-08-15T17:58:36.280114+0000","last_active":"2024-08-15T17:58:36.285495+0000","last_peered":"2024-08-15T17:58:36.285495+0000","last_clean":"2024-08-15T17:58:36.285495+0000","last_became_active":"2024-08-15T17:58:36.279785+0000","last_became_peered":"2024-08-15T17:58:36.279785+0000","last_unstale":"2024-08-15T17:58:36.285495+0000","last_undegraded":"2024-08-15T17:58:36.285495+0000","last_fullsized":"2024-08-15T17:58:36.285495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:56:03.150852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.292089+0000","last_change":"2024-08-15T17:58:36.290521+0000","last_active":"2024-08-15T17:58:36.292089+0000","last_peered":"2024-08-15T17:58:36.292089+0000","last_clean":"2024-08-15T17:58:36.292089+0000","last_became_active":"2024-08-15T17:58:36.290154+0000","last_became_peered":"2024-08-15T17:58:36.290154+0000","last_unstale":"2024-08-15T17:58:36.292089+0000","last_undegraded":"2024-08-15T17:58:36.292089+0000","last_fullsized":"2024-08-15T17:58:36.292089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:45.183805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.296070+0000","last_change":"2024-08-15T17:58:36.285210+0000","last_active":"2024-08-15T17:58:36.296070+0000","last_peered":"2024-08-15T17:58:36.296070+0000","last_clean":"2024-08-15T17:58:36.296070+0000","last_became_active":"2024-08-15T17:58:36.284752+0000","last_became_peered":"2024-08-15T17:58:36.284752+0000","last_unstale":"2024-08-15T17:58:36.296070+0000","last_undegraded":"2024-08-15T17:58:36.296070+0000","last_fullsized":"2024-08-15T17:58:36.296070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:18:07.754382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.292174+0000","last_change":"2024-08-15T17:58:36.285554+0000","last_active":"2024-08-15T17:58:36.292174+0000","last_peered":"2024-08-15T17:58:36.292174+0000","last_clean":"2024-08-15T17:58:36.292174+0000","last_became_active":"2024-08-15T17:58:36.285281+0000","last_became_peered":"2024-08-15T17:58:36.285281+0000","last_unstale":"2024-08-15T17:58:36.292174+0000","last_undegraded":"2024-08-15T17:58:36.292174+0000","last_fullsized":"2024-08-15T17:58:36.292174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:12:39.229320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290783+0000","last_change":"2024-08-15T17:58:36.280839+0000","last_active":"2024-08-15T17:58:36.290783+0000","last_peered":"2024-08-15T17:58:36.290783+0000","last_clean":"2024-08-15T17:58:36.290783+0000","last_became_active":"2024-08-15T17:58:36.280519+0000","last_became_peered":"2024-08-15T17:58:36.280519+0000","last_unstale":"2024-08-15T17:58:36.290783+0000","last_undegraded":"2024-08-15T17:58:36.290783+0000","last_fullsized":"2024-08-15T17:58:36.290783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.281236+0000","last_change":"2024-08-15T17:58:36.278583+0000","last_active":"2024-08-15T17:58:36.281236+0000","last_peered":"2024-08-15T17:58:36.281236+0000","last_clean":"2024-08-15T17:58:36.281236+0000","last_became_active":"2024-08-15T17:58:36.278308+0000","last_became_peered":"2024-08-15T17:58:36.278308+0000","last_unstale":"2024-08-15T17:58:36.281236+0000","last_undegraded":"2024-08-15T17:58:36.281236+0000","last_fullsized":"2024-08-15T17:58:36.281236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:56:49.069407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286580+0000","last_change":"2024-08-15T17:58:36.283003+0000","last_active":"2024-08-15T17:58:36.286580+0000","last_peered":"2024-08-15T17:58:36.286580+0000","last_clean":"2024-08-15T17:58:36.286580+0000","last_became_active":"2024-08-15T17:58:36.282691+0000","last_became_peered":"2024-08-15T17:58:36.282691+0000","last_unstale":"2024-08-15T17:58:36.286580+0000","last_undegraded":"2024-08-15T17:58:36.286580+0000","last_fullsized":"2024-08-15T17:58:36.286580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:48:28.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288825+0000","last_change":"2024-08-15T17:58:36.285479+0000","last_active":"2024-08-15T17:58:36.288825+0000","last_peered":"2024-08-15T17:58:36.288825+0000","last_clean":"2024-08-15T17:58:36.288825+0000","last_became_active":"2024-08-15T17:58:36.285044+0000","last_became_peered":"2024-08-15T17:58:36.285044+0000","last_unstale":"2024-08-15T17:58:36.288825+0000","last_undegraded":"2024-08-15T17:58:36.288825+0000","last_fullsized":"2024-08-15T17:58:36.288825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:55:54.644188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286608+0000","last_change":"2024-08-15T17:58:36.284044+0000","last_active":"2024-08-15T17:58:36.286608+0000","last_peered":"2024-08-15T17:58:36.286608+0000","last_clean":"2024-08-15T17:58:36.286608+0000","last_became_active":"2024-08-15T17:58:36.283664+0000","last_became_peered":"2024-08-15T17:58:36.283664+0000","last_unstale":"2024-08-15T17:58:36.286608+0000","last_undegraded":"2024-08-15T17:58:36.286608+0000","last_fullsized":"2024-08-15T17:58:36.286608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:58:52.618737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294474+0000","last_change":"2024-08-15T17:58:36.290152+0000","last_active":"2024-08-15T17:58:36.294474+0000","last_peered":"2024-08-15T17:58:36.294474+0000","last_clean":"2024-08-15T17:58:36.294474+0000","last_became_active":"2024-08-15T17:58:36.287149+0000","last_became_peered":"2024-08-15T17:58:36.287149+0000","last_unstale":"2024-08-15T17:58:36.294474+0000","last_undegraded":"2024-08-15T17:58:36.294474+0000","last_fullsized":"2024-08-15T17:58:36.294474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:50:39.313237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289403+0000","last_change":"2024-08-15T17:58:36.287149+0000","last_active":"2024-08-15T17:58:36.289403+0000","last_peered":"2024-08-15T17:58:36.289403+0000","last_clean":"2024-08-15T17:58:36.289403+0000","last_became_active":"2024-08-15T17:58:36.286239+0000","last_became_peered":"2024-08-15T17:58:36.286239+0000","last_unstale":"2024-08-15T17:58:36.289403+0000","last_undegraded":"2024-08-15T17:58:36.289403+0000","last_fullsized":"2024-08-15T17:58:36.289403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:02:13.223698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286778+0000","last_change":"2024-08-15T17:58:36.283315+0000","last_active":"2024-08-15T17:58:36.286778+0000","last_peered":"2024-08-15T17:58:36.286778+0000","last_clean":"2024-08-15T17:58:36.286778+0000","last_became_active":"2024-08-15T17:58:36.282873+0000","last_became_peered":"2024-08-15T17:58:36.282873+0000","last_unstale":"2024-08-15T17:58:36.286778+0000","last_undegraded":"2024-08-15T17:58:36.286778+0000","last_fullsized":"2024-08-15T17:58:36.286778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:31:12.400383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.290819+0000","last_change":"2024-08-15T17:58:36.284677+0000","last_active":"2024-08-15T17:58:36.290819+0000","last_peered":"2024-08-15T17:58:36.290819+0000","last_clean":"2024-08-15T17:58:36.290819+0000","last_became_active":"2024-08-15T17:58:36.281734+0000","last_became_peered":"2024-08-15T17:58:36.281734+0000","last_unstale":"2024-08-15T17:58:36.290819+0000","last_undegraded":"2024-08-15T17:58:36.290819+0000","last_fullsized":"2024-08-15T17:58:36.290819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288562+0000","last_change":"2024-08-15T17:58:36.281437+0000","last_active":"2024-08-15T17:58:36.288562+0000","last_peered":"2024-08-15T17:58:36.288562+0000","last_clean":"2024-08-15T17:58:36.288562+0000","last_became_active":"2024-08-15T17:58:36.280874+0000","last_became_peered":"2024-08-15T17:58:36.280874+0000","last_unstale":"2024-08-15T17:58:36.288562+0000","last_undegraded":"2024-08-15T17:58:36.288562+0000","last_fullsized":"2024-08-15T17:58:36.288562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:05.015208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288535+0000","last_change":"2024-08-15T17:58:36.285476+0000","last_active":"2024-08-15T17:58:36.288535+0000","last_peered":"2024-08-15T17:58:36.288535+0000","last_clean":"2024-08-15T17:58:36.288535+0000","last_became_active":"2024-08-15T17:58:36.285049+0000","last_became_peered":"2024-08-15T17:58:36.285049+0000","last_unstale":"2024-08-15T17:58:36.288535+0000","last_undegraded":"2024-08-15T17:58:36.288535+0000","last_fullsized":"2024-08-15T17:58:36.288535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:32:58.557897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294340+0000","last_change":"2024-08-15T17:58:36.286646+0000","last_active":"2024-08-15T17:58:36.294340+0000","last_peered":"2024-08-15T17:58:36.294340+0000","last_clean":"2024-08-15T17:58:36.294340+0000","last_became_active":"2024-08-15T17:58:36.286375+0000","last_became_peered":"2024-08-15T17:58:36.286375+0000","last_unstale":"2024-08-15T17:58:36.294340+0000","last_undegraded":"2024-08-15T17:58:36.294340+0000","last_fullsized":"2024-08-15T17:58:36.294340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:33.556488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285304+0000","last_change":"2024-08-15T17:58:36.278354+0000","last_active":"2024-08-15T17:58:36.285304+0000","last_peered":"2024-08-15T17:58:36.285304+0000","last_clean":"2024-08-15T17:58:36.285304+0000","last_became_active":"2024-08-15T17:58:36.278061+0000","last_became_peered":"2024-08-15T17:58:36.278061+0000","last_unstale":"2024-08-15T17:58:36.285304+0000","last_undegraded":"2024-08-15T17:58:36.285304+0000","last_fullsized":"2024-08-15T17:58:36.285304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:45.183805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301650+0000","last_change":"2024-08-15T17:58:35.277170+0000","last_active":"2024-08-15T17:58:36.301650+0000","last_peered":"2024-08-15T17:58:36.301650+0000","last_clean":"2024-08-15T17:58:36.301650+0000","last_became_active":"2024-08-15T17:58:35.276904+0000","last_became_peered":"2024-08-15T17:58:35.276904+0000","last_unstale":"2024-08-15T17:58:36.301650+0000","last_undegraded":"2024-08-15T17:58:36.301650+0000","last_fullsized":"2024-08-15T17:58:36.301650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:11:34.747972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.274623+0000","last_change":"2024-08-15T17:58:36.271912+0000","last_active":"2024-08-15T17:58:36.274623+0000","last_peered":"2024-08-15T17:58:36.274623+0000","last_clean":"2024-08-15T17:58:36.274623+0000","last_became_active":"2024-08-15T17:58:36.271346+0000","last_became_peered":"2024-08-15T17:58:36.271346+0000","last_unstale":"2024-08-15T17:58:36.274623+0000","last_undegraded":"2024-08-15T17:58:36.274623+0000","last_fullsized":"2024-08-15T17:58:36.274623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:54:02.548772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298247+0000","last_change":"2024-08-15T17:58:35.266446+0000","last_active":"2024-08-15T17:58:36.298247+0000","last_peered":"2024-08-15T17:58:36.298247+0000","last_clean":"2024-08-15T17:58:36.298247+0000","last_became_active":"2024-08-15T17:58:35.265867+0000","last_became_peered":"2024-08-15T17:58:35.265867+0000","last_unstale":"2024-08-15T17:58:36.298247+0000","last_undegraded":"2024-08-15T17:58:36.298247+0000","last_fullsized":"2024-08-15T17:58:36.298247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:11:26.241336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.280182+0000","last_change":"2024-08-15T17:58:35.275168+0000","last_active":"2024-08-15T17:58:36.280182+0000","last_peered":"2024-08-15T17:58:36.280182+0000","last_clean":"2024-08-15T17:58:36.280182+0000","last_became_active":"2024-08-15T17:58:35.274863+0000","last_became_peered":"2024-08-15T17:58:35.274863+0000","last_unstale":"2024-08-15T17:58:36.280182+0000","last_undegraded":"2024-08-15T17:58:36.280182+0000","last_fullsized":"2024-08-15T17:58:36.280182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:17:09.213636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293638+0000","last_change":"2024-08-15T17:58:36.290048+0000","last_active":"2024-08-15T17:58:36.293638+0000","last_peered":"2024-08-15T17:58:36.293638+0000","last_clean":"2024-08-15T17:58:36.293638+0000","last_became_active":"2024-08-15T17:58:36.289720+0000","last_became_peered":"2024-08-15T17:58:36.289720+0000","last_unstale":"2024-08-15T17:58:36.293638+0000","last_undegraded":"2024-08-15T17:58:36.293638+0000","last_fullsized":"2024-08-15T17:58:36.293638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:55.518837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.272221+0000","last_change":"2024-08-15T17:58:35.266688+0000","last_active":"2024-08-15T17:58:36.272221+0000","last_peered":"2024-08-15T17:58:36.272221+0000","last_clean":"2024-08-15T17:58:36.272221+0000","last_became_active":"2024-08-15T17:58:35.266306+0000","last_became_peered":"2024-08-15T17:58:35.266306+0000","last_unstale":"2024-08-15T17:58:36.272221+0000","last_undegraded":"2024-08-15T17:58:36.272221+0000","last_fullsized":"2024-08-15T17:58:36.272221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:11:34.747972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289814+0000","last_change":"2024-08-15T17:58:36.287582+0000","last_active":"2024-08-15T17:58:36.289814+0000","last_peered":"2024-08-15T17:58:36.289814+0000","last_clean":"2024-08-15T17:58:36.289814+0000","last_became_active":"2024-08-15T17:58:36.286980+0000","last_became_peered":"2024-08-15T17:58:36.286980+0000","last_unstale":"2024-08-15T17:58:36.289814+0000","last_undegraded":"2024-08-15T17:58:36.289814+0000","last_fullsized":"2024-08-15T17:58:36.289814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:49:19.919158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285974+0000","last_change":"2024-08-15T17:58:35.267834+0000","last_active":"2024-08-15T17:58:36.285974+0000","last_peered":"2024-08-15T17:58:36.285974+0000","last_clean":"2024-08-15T17:58:36.285974+0000","last_became_active":"2024-08-15T17:58:35.267249+0000","last_became_peered":"2024-08-15T17:58:35.267249+0000","last_unstale":"2024-08-15T17:58:36.285974+0000","last_undegraded":"2024-08-15T17:58:36.285974+0000","last_fullsized":"2024-08-15T17:58:36.285974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:17.590038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288210+0000","last_change":"2024-08-15T17:58:36.280965+0000","last_active":"2024-08-15T17:58:36.288210+0000","last_peered":"2024-08-15T17:58:36.288210+0000","last_clean":"2024-08-15T17:58:36.288210+0000","last_became_active":"2024-08-15T17:58:36.280684+0000","last_became_peered":"2024-08-15T17:58:36.280684+0000","last_unstale":"2024-08-15T17:58:36.288210+0000","last_undegraded":"2024-08-15T17:58:36.288210+0000","last_fullsized":"2024-08-15T17:58:36.288210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:15:59.759500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.274432+0000","last_change":"2024-08-15T17:58:35.268261+0000","last_active":"2024-08-15T17:58:36.274432+0000","last_peered":"2024-08-15T17:58:36.274432+0000","last_clean":"2024-08-15T17:58:36.274432+0000","last_became_active":"2024-08-15T17:58:35.266109+0000","last_became_peered":"2024-08-15T17:58:35.266109+0000","last_unstale":"2024-08-15T17:58:36.274432+0000","last_undegraded":"2024-08-15T17:58:36.274432+0000","last_fullsized":"2024-08-15T17:58:36.274432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:06:34.826184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289339+0000","last_change":"2024-08-15T17:58:36.283639+0000","last_active":"2024-08-15T17:58:36.289339+0000","last_peered":"2024-08-15T17:58:36.289339+0000","last_clean":"2024-08-15T17:58:36.289339+0000","last_became_active":"2024-08-15T17:58:36.282343+0000","last_became_peered":"2024-08-15T17:58:36.282343+0000","last_unstale":"2024-08-15T17:58:36.289339+0000","last_undegraded":"2024-08-15T17:58:36.289339+0000","last_fullsized":"2024-08-15T17:58:36.289339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:58:52.618737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301091+0000","last_change":"2024-08-15T17:58:35.276856+0000","last_active":"2024-08-15T17:58:36.301091+0000","last_peered":"2024-08-15T17:58:36.301091+0000","last_clean":"2024-08-15T17:58:36.301091+0000","last_became_active":"2024-08-15T17:58:35.275745+0000","last_became_peered":"2024-08-15T17:58:35.275745+0000","last_unstale":"2024-08-15T17:58:36.301091+0000","last_undegraded":"2024-08-15T17:58:36.301091+0000","last_fullsized":"2024-08-15T17:58:36.301091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:26:33.980860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.279341+0000","last_change":"2024-08-15T17:58:36.272759+0000","last_active":"2024-08-15T17:58:36.279341+0000","last_peered":"2024-08-15T17:58:36.279341+0000","last_clean":"2024-08-15T17:58:36.279341+0000","last_became_active":"2024-08-15T17:58:36.271418+0000","last_became_peered":"2024-08-15T17:58:36.271418+0000","last_unstale":"2024-08-15T17:58:36.279341+0000","last_undegraded":"2024-08-15T17:58:36.279341+0000","last_fullsized":"2024-08-15T17:58:36.279341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:32:58.557897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301498+0000","last_change":"2024-08-15T17:58:35.271651+0000","last_active":"2024-08-15T17:58:36.301498+0000","last_peered":"2024-08-15T17:58:36.301498+0000","last_clean":"2024-08-15T17:58:36.301498+0000","last_became_active":"2024-08-15T17:58:35.271306+0000","last_became_peered":"2024-08-15T17:58:35.271306+0000","last_unstale":"2024-08-15T17:58:36.301498+0000","last_undegraded":"2024-08-15T17:58:36.301498+0000","last_fullsized":"2024-08-15T17:58:36.301498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:44.174826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.296030+0000","last_change":"2024-08-15T17:58:36.284247+0000","last_active":"2024-08-15T17:58:36.296030+0000","last_peered":"2024-08-15T17:58:36.296030+0000","last_clean":"2024-08-15T17:58:36.296030+0000","last_became_active":"2024-08-15T17:58:36.283981+0000","last_became_peered":"2024-08-15T17:58:36.283981+0000","last_unstale":"2024-08-15T17:58:36.296030+0000","last_undegraded":"2024-08-15T17:58:36.296030+0000","last_fullsized":"2024-08-15T17:58:36.296030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:09:08.360709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.305587+0000","last_change":"2024-08-15T17:58:35.268853+0000","last_active":"2024-08-15T17:58:36.305587+0000","last_peered":"2024-08-15T17:58:36.305587+0000","last_clean":"2024-08-15T17:58:36.305587+0000","last_became_active":"2024-08-15T17:58:35.268541+0000","last_became_peered":"2024-08-15T17:58:35.268541+0000","last_unstale":"2024-08-15T17:58:36.305587+0000","last_undegraded":"2024-08-15T17:58:36.305587+0000","last_fullsized":"2024-08-15T17:58:36.305587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:33:31.329931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288269+0000","last_change":"2024-08-15T17:58:36.278321+0000","last_active":"2024-08-15T17:58:36.288269+0000","last_peered":"2024-08-15T17:58:36.288269+0000","last_clean":"2024-08-15T17:58:36.288269+0000","last_became_active":"2024-08-15T17:58:36.278014+0000","last_became_peered":"2024-08-15T17:58:36.278014+0000","last_unstale":"2024-08-15T17:58:36.288269+0000","last_undegraded":"2024-08-15T17:58:36.288269+0000","last_fullsized":"2024-08-15T17:58:36.288269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:27:54.650775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.274096+0000","last_change":"2024-08-15T17:58:35.278113+0000","last_active":"2024-08-15T17:58:36.274096+0000","last_peered":"2024-08-15T17:58:36.274096+0000","last_clean":"2024-08-15T17:58:36.274096+0000","last_became_active":"2024-08-15T17:58:35.277811+0000","last_became_peered":"2024-08-15T17:58:35.277811+0000","last_unstale":"2024-08-15T17:58:36.274096+0000","last_undegraded":"2024-08-15T17:58:36.274096+0000","last_fullsized":"2024-08-15T17:58:36.274096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:12:40.711380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294826+0000","last_change":"2024-08-15T17:58:36.283999+0000","last_active":"2024-08-15T17:58:36.294826+0000","last_peered":"2024-08-15T17:58:36.294826+0000","last_clean":"2024-08-15T17:58:36.294826+0000","last_became_active":"2024-08-15T17:58:36.283723+0000","last_became_peered":"2024-08-15T17:58:36.283723+0000","last_unstale":"2024-08-15T17:58:36.294826+0000","last_undegraded":"2024-08-15T17:58:36.294826+0000","last_fullsized":"2024-08-15T17:58:36.294826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:23:04.074155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.283536+0000","last_change":"2024-08-15T17:58:35.268112+0000","last_active":"2024-08-15T17:58:36.283536+0000","last_peered":"2024-08-15T17:58:36.283536+0000","last_clean":"2024-08-15T17:58:36.283536+0000","last_became_active":"2024-08-15T17:58:35.265298+0000","last_became_peered":"2024-08-15T17:58:35.265298+0000","last_unstale":"2024-08-15T17:58:36.283536+0000","last_undegraded":"2024-08-15T17:58:36.283536+0000","last_fullsized":"2024-08-15T17:58:36.283536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:13.047239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294320+0000","last_change":"2024-08-15T17:58:36.286548+0000","last_active":"2024-08-15T17:58:36.294320+0000","last_peered":"2024-08-15T17:58:36.294320+0000","last_clean":"2024-08-15T17:58:36.294320+0000","last_became_active":"2024-08-15T17:58:36.286287+0000","last_became_peered":"2024-08-15T17:58:36.286287+0000","last_unstale":"2024-08-15T17:58:36.294320+0000","last_undegraded":"2024-08-15T17:58:36.294320+0000","last_fullsized":"2024-08-15T17:58:36.294320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:07:03.605391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.304604+0000","last_change":"2024-08-15T17:58:35.271260+0000","last_active":"2024-08-15T17:58:36.304604+0000","last_peered":"2024-08-15T17:58:36.304604+0000","last_clean":"2024-08-15T17:58:36.304604+0000","last_became_active":"2024-08-15T17:58:35.270816+0000","last_became_peered":"2024-08-15T17:58:35.270816+0000","last_unstale":"2024-08-15T17:58:36.304604+0000","last_undegraded":"2024-08-15T17:58:36.304604+0000","last_fullsized":"2024-08-15T17:58:36.304604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:58:46.815165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.291658+0000","last_change":"2024-08-15T17:58:36.283169+0000","last_active":"2024-08-15T17:58:36.291658+0000","last_peered":"2024-08-15T17:58:36.291658+0000","last_clean":"2024-08-15T17:58:36.291658+0000","last_became_active":"2024-08-15T17:58:36.282882+0000","last_became_peered":"2024-08-15T17:58:36.282882+0000","last_unstale":"2024-08-15T17:58:36.291658+0000","last_undegraded":"2024-08-15T17:58:36.291658+0000","last_fullsized":"2024-08-15T17:58:36.291658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:24:27.743039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.299612+0000","last_change":"2024-08-15T17:58:35.266260+0000","last_active":"2024-08-15T17:58:36.299612+0000","last_peered":"2024-08-15T17:58:36.299612+0000","last_clean":"2024-08-15T17:58:36.299612+0000","last_became_active":"2024-08-15T17:58:35.265941+0000","last_became_peered":"2024-08-15T17:58:35.265941+0000","last_unstale":"2024-08-15T17:58:36.299612+0000","last_undegraded":"2024-08-15T17:58:36.299612+0000","last_fullsized":"2024-08-15T17:58:36.299612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:48:09.608873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.296591+0000","last_change":"2024-08-15T17:58:36.285123+0000","last_active":"2024-08-15T17:58:36.296591+0000","last_peered":"2024-08-15T17:58:36.296591+0000","last_clean":"2024-08-15T17:58:36.296591+0000","last_became_active":"2024-08-15T17:58:36.284782+0000","last_became_peered":"2024-08-15T17:58:36.284782+0000","last_unstale":"2024-08-15T17:58:36.296591+0000","last_undegraded":"2024-08-15T17:58:36.296591+0000","last_fullsized":"2024-08-15T17:58:36.296591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:42:52.458542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301625+0000","last_change":"2024-08-15T17:58:35.273755+0000","last_active":"2024-08-15T17:58:36.301625+0000","last_peered":"2024-08-15T17:58:36.301625+0000","last_clean":"2024-08-15T17:58:36.301625+0000","last_became_active":"2024-08-15T17:58:35.273318+0000","last_became_peered":"2024-08-15T17:58:35.273318+0000","last_unstale":"2024-08-15T17:58:36.301625+0000","last_undegraded":"2024-08-15T17:58:36.301625+0000","last_fullsized":"2024-08-15T17:58:36.301625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:13.047239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.295367+0000","last_change":"2024-08-15T17:58:36.286635+0000","last_active":"2024-08-15T17:58:36.295367+0000","last_peered":"2024-08-15T17:58:36.295367+0000","last_clean":"2024-08-15T17:58:36.295367+0000","last_became_active":"2024-08-15T17:58:36.286062+0000","last_became_peered":"2024-08-15T17:58:36.286062+0000","last_unstale":"2024-08-15T17:58:36.295367+0000","last_undegraded":"2024-08-15T17:58:36.295367+0000","last_fullsized":"2024-08-15T17:58:36.295367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:21:42.887962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.302921+0000","last_change":"2024-08-15T17:58:35.274846+0000","last_active":"2024-08-15T17:58:36.302921+0000","last_peered":"2024-08-15T17:58:36.302921+0000","last_clean":"2024-08-15T17:58:36.302921+0000","last_became_active":"2024-08-15T17:58:35.274543+0000","last_became_peered":"2024-08-15T17:58:35.274543+0000","last_unstale":"2024-08-15T17:58:36.302921+0000","last_undegraded":"2024-08-15T17:58:36.302921+0000","last_fullsized":"2024-08-15T17:58:36.302921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:11:26.241336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288832+0000","last_change":"2024-08-15T17:58:36.285388+0000","last_active":"2024-08-15T17:58:36.288832+0000","last_peered":"2024-08-15T17:58:36.288832+0000","last_clean":"2024-08-15T17:58:36.288832+0000","last_became_active":"2024-08-15T17:58:36.285136+0000","last_became_peered":"2024-08-15T17:58:36.285136+0000","last_unstale":"2024-08-15T17:58:36.288832+0000","last_undegraded":"2024-08-15T17:58:36.288832+0000","last_fullsized":"2024-08-15T17:58:36.288832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:43.158602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.273480+0000","last_change":"2024-08-15T17:58:35.272166+0000","last_active":"2024-08-15T17:58:36.273480+0000","last_peered":"2024-08-15T17:58:36.273480+0000","last_clean":"2024-08-15T17:58:36.273480+0000","last_became_active":"2024-08-15T17:58:35.271841+0000","last_became_peered":"2024-08-15T17:58:35.271841+0000","last_unstale":"2024-08-15T17:58:36.273480+0000","last_undegraded":"2024-08-15T17:58:36.273480+0000","last_fullsized":"2024-08-15T17:58:36.273480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:40:09.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.296052+0000","last_change":"2024-08-15T17:58:36.285039+0000","last_active":"2024-08-15T17:58:36.296052+0000","last_peered":"2024-08-15T17:58:36.296052+0000","last_clean":"2024-08-15T17:58:36.296052+0000","last_became_active":"2024-08-15T17:58:36.284458+0000","last_became_peered":"2024-08-15T17:58:36.284458+0000","last_unstale":"2024-08-15T17:58:36.296052+0000","last_undegraded":"2024-08-15T17:58:36.296052+0000","last_fullsized":"2024-08-15T17:58:36.296052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:37:55.417330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.271012+0000","last_change":"2024-08-15T17:58:24.394014+0000","last_active":"2024-08-15T17:58:36.271012+0000","last_peered":"2024-08-15T17:58:36.271012+0000","last_clean":"2024-08-15T17:58:36.271012+0000","last_became_active":"2024-08-15T17:58:24.391735+0000","last_became_peered":"2024-08-15T17:58:24.391735+0000","last_unstale":"2024-08-15T17:58:36.271012+0000","last_undegraded":"2024-08-15T17:58:36.271012+0000","last_fullsized":"2024-08-15T17:58:36.271012+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_clean_scrub_stamp":"2024-08-15T17:58:22.996335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:31.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.273936+0000","last_change":"2024-08-15T17:58:35.269765+0000","last_active":"2024-08-15T17:58:36.273936+0000","last_peered":"2024-08-15T17:58:36.273936+0000","last_clean":"2024-08-15T17:58:36.273936+0000","last_became_active":"2024-08-15T17:58:35.269249+0000","last_became_peered":"2024-08-15T17:58:35.269249+0000","last_unstale":"2024-08-15T17:58:36.273936+0000","last_undegraded":"2024-08-15T17:58:36.273936+0000","last_fullsized":"2024-08-15T17:58:36.273936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:27:34.522329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.292584+0000","last_change":"2024-08-15T17:58:36.286437+0000","last_active":"2024-08-15T17:58:36.292584+0000","last_peered":"2024-08-15T17:58:36.292584+0000","last_clean":"2024-08-15T17:58:36.292584+0000","last_became_active":"2024-08-15T17:58:36.285909+0000","last_became_peered":"2024-08-15T17:58:36.285909+0000","last_unstale":"2024-08-15T17:58:36.292584+0000","last_undegraded":"2024-08-15T17:58:36.292584+0000","last_fullsized":"2024-08-15T17:58:36.292584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:47:17.486762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.270935+0000","last_change":"2024-08-15T17:58:35.275547+0000","last_active":"2024-08-15T17:58:36.270935+0000","last_peered":"2024-08-15T17:58:36.270935+0000","last_clean":"2024-08-15T17:58:36.270935+0000","last_became_active":"2024-08-15T17:58:35.275253+0000","last_became_peered":"2024-08-15T17:58:35.275253+0000","last_unstale":"2024-08-15T17:58:36.270935+0000","last_undegraded":"2024-08-15T17:58:36.270935+0000","last_fullsized":"2024-08-15T17:58:36.270935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:44.174826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288317+0000","last_change":"2024-08-15T17:58:36.277792+0000","last_active":"2024-08-15T17:58:36.288317+0000","last_peered":"2024-08-15T17:58:36.288317+0000","last_clean":"2024-08-15T17:58:36.288317+0000","last_became_active":"2024-08-15T17:58:36.276715+0000","last_became_peered":"2024-08-15T17:58:36.276715+0000","last_unstale":"2024-08-15T17:58:36.288317+0000","last_undegraded":"2024-08-15T17:58:36.288317+0000","last_fullsized":"2024-08-15T17:58:36.288317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:59:27.743235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.275699+0000","last_change":"2024-08-15T17:58:35.280834+0000","last_active":"2024-08-15T17:58:36.275699+0000","last_peered":"2024-08-15T17:58:36.275699+0000","last_clean":"2024-08-15T17:58:36.275699+0000","last_became_active":"2024-08-15T17:58:35.280524+0000","last_became_peered":"2024-08-15T17:58:35.280524+0000","last_unstale":"2024-08-15T17:58:36.275699+0000","last_undegraded":"2024-08-15T17:58:36.275699+0000","last_fullsized":"2024-08-15T17:58:36.275699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:00:50.031787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289140+0000","last_change":"2024-08-15T17:58:36.283772+0000","last_active":"2024-08-15T17:58:36.289140+0000","last_peered":"2024-08-15T17:58:36.289140+0000","last_clean":"2024-08-15T17:58:36.289140+0000","last_became_active":"2024-08-15T17:58:36.283482+0000","last_became_peered":"2024-08-15T17:58:36.283482+0000","last_unstale":"2024-08-15T17:58:36.289140+0000","last_undegraded":"2024-08-15T17:58:36.289140+0000","last_fullsized":"2024-08-15T17:58:36.289140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:33.556488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.274203+0000","last_change":"2024-08-15T17:58:35.270088+0000","last_active":"2024-08-15T17:58:36.274203+0000","last_peered":"2024-08-15T17:58:36.274203+0000","last_clean":"2024-08-15T17:58:36.274203+0000","last_became_active":"2024-08-15T17:58:35.268586+0000","last_became_peered":"2024-08-15T17:58:35.268586+0000","last_unstale":"2024-08-15T17:58:36.274203+0000","last_undegraded":"2024-08-15T17:58:36.274203+0000","last_fullsized":"2024-08-15T17:58:36.274203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:17:09.213636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.285313+0000","last_change":"2024-08-15T17:58:36.277552+0000","last_active":"2024-08-15T17:58:36.285313+0000","last_peered":"2024-08-15T17:58:36.285313+0000","last_clean":"2024-08-15T17:58:36.285313+0000","last_became_active":"2024-08-15T17:58:36.277173+0000","last_became_peered":"2024-08-15T17:58:36.277173+0000","last_unstale":"2024-08-15T17:58:36.285313+0000","last_undegraded":"2024-08-15T17:58:36.285313+0000","last_fullsized":"2024-08-15T17:58:36.285313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:18.599017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.272956+0000","last_change":"2024-08-15T17:58:35.270974+0000","last_active":"2024-08-15T17:58:36.272956+0000","last_peered":"2024-08-15T17:58:36.272956+0000","last_clean":"2024-08-15T17:58:36.272956+0000","last_became_active":"2024-08-15T17:58:35.270645+0000","last_became_peered":"2024-08-15T17:58:35.270645+0000","last_unstale":"2024-08-15T17:58:36.272956+0000","last_undegraded":"2024-08-15T17:58:36.272956+0000","last_fullsized":"2024-08-15T17:58:36.272956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:06:44.174826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288251+0000","last_change":"2024-08-15T17:58:36.282072+0000","last_active":"2024-08-15T17:58:36.288251+0000","last_peered":"2024-08-15T17:58:36.288251+0000","last_clean":"2024-08-15T17:58:36.288251+0000","last_became_active":"2024-08-15T17:58:36.281850+0000","last_became_peered":"2024-08-15T17:58:36.281850+0000","last_unstale":"2024-08-15T17:58:36.288251+0000","last_undegraded":"2024-08-15T17:58:36.288251+0000","last_fullsized":"2024-08-15T17:58:36.288251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:54:56.996071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.277028+0000","last_change":"2024-08-15T17:58:35.275830+0000","last_active":"2024-08-15T17:58:36.277028+0000","last_peered":"2024-08-15T17:58:36.277028+0000","last_clean":"2024-08-15T17:58:36.277028+0000","last_became_active":"2024-08-15T17:58:35.275532+0000","last_became_peered":"2024-08-15T17:58:35.275532+0000","last_unstale":"2024-08-15T17:58:36.277028+0000","last_undegraded":"2024-08-15T17:58:36.277028+0000","last_fullsized":"2024-08-15T17:58:36.277028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:38.845281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289260+0000","last_change":"2024-08-15T17:58:36.284264+0000","last_active":"2024-08-15T17:58:36.289260+0000","last_peered":"2024-08-15T17:58:36.289260+0000","last_clean":"2024-08-15T17:58:36.289260+0000","last_became_active":"2024-08-15T17:58:36.283964+0000","last_became_peered":"2024-08-15T17:58:36.283964+0000","last_unstale":"2024-08-15T17:58:36.289260+0000","last_undegraded":"2024-08-15T17:58:36.289260+0000","last_fullsized":"2024-08-15T17:58:36.289260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:07:03.605391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-15T17:58:37.297038+0000","last_change":"2024-08-15T17:58:35.274546+0000","last_active":"2024-08-15T17:58:37.297038+0000","last_peered":"2024-08-15T17:58:37.297038+0000","last_clean":"2024-08-15T17:58:37.297038+0000","last_became_active":"2024-08-15T17:58:35.274232+0000","last_became_peered":"2024-08-15T17:58:35.274232+0000","last_unstale":"2024-08-15T17:58:37.297038+0000","last_undegraded":"2024-08-15T17:58:37.297038+0000","last_fullsized":"2024-08-15T17:58:37.297038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:17.590038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293817+0000","last_change":"2024-08-15T17:58:36.290821+0000","last_active":"2024-08-15T17:58:36.293817+0000","last_peered":"2024-08-15T17:58:36.293817+0000","last_clean":"2024-08-15T17:58:36.293817+0000","last_became_active":"2024-08-15T17:58:36.290495+0000","last_became_peered":"2024-08-15T17:58:36.290495+0000","last_unstale":"2024-08-15T17:58:36.293817+0000","last_undegraded":"2024-08-15T17:58:36.293817+0000","last_fullsized":"2024-08-15T17:58:36.293817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:43.158602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298969+0000","last_change":"2024-08-15T17:58:35.274034+0000","last_active":"2024-08-15T17:58:36.298969+0000","last_peered":"2024-08-15T17:58:36.298969+0000","last_clean":"2024-08-15T17:58:36.298969+0000","last_became_active":"2024-08-15T17:58:35.273028+0000","last_became_peered":"2024-08-15T17:58:35.273028+0000","last_unstale":"2024-08-15T17:58:36.298969+0000","last_undegraded":"2024-08-15T17:58:36.298969+0000","last_fullsized":"2024-08-15T17:58:36.298969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:26:33.980860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.289390+0000","last_change":"2024-08-15T17:58:36.284750+0000","last_active":"2024-08-15T17:58:36.289390+0000","last_peered":"2024-08-15T17:58:36.289390+0000","last_clean":"2024-08-15T17:58:36.289390+0000","last_became_active":"2024-08-15T17:58:36.284480+0000","last_became_peered":"2024-08-15T17:58:36.284480+0000","last_unstale":"2024-08-15T17:58:36.289390+0000","last_undegraded":"2024-08-15T17:58:36.289390+0000","last_fullsized":"2024-08-15T17:58:36.289390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:08:55.518837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.270849+0000","last_change":"2024-08-15T17:58:35.272396+0000","last_active":"2024-08-15T17:58:36.270849+0000","last_peered":"2024-08-15T17:58:36.270849+0000","last_clean":"2024-08-15T17:58:36.270849+0000","last_became_active":"2024-08-15T17:58:35.272141+0000","last_became_peered":"2024-08-15T17:58:35.272141+0000","last_unstale":"2024-08-15T17:58:36.270849+0000","last_undegraded":"2024-08-15T17:58:36.270849+0000","last_fullsized":"2024-08-15T17:58:36.270849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:27:17.590038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.286071+0000","last_change":"2024-08-15T17:58:36.278104+0000","last_active":"2024-08-15T17:58:36.286071+0000","last_peered":"2024-08-15T17:58:36.286071+0000","last_clean":"2024-08-15T17:58:36.286071+0000","last_became_active":"2024-08-15T17:58:36.277844+0000","last_became_peered":"2024-08-15T17:58:36.277844+0000","last_unstale":"2024-08-15T17:58:36.286071+0000","last_undegraded":"2024-08-15T17:58:36.286071+0000","last_fullsized":"2024-08-15T17:58:36.286071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:26:34.989839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.275661+0000","last_change":"2024-08-15T17:58:35.277768+0000","last_active":"2024-08-15T17:58:36.275661+0000","last_peered":"2024-08-15T17:58:36.275661+0000","last_clean":"2024-08-15T17:58:36.275661+0000","last_became_active":"2024-08-15T17:58:35.277217+0000","last_became_peered":"2024-08-15T17:58:35.277217+0000","last_unstale":"2024-08-15T17:58:36.275661+0000","last_undegraded":"2024-08-15T17:58:36.275661+0000","last_fullsized":"2024-08-15T17:58:36.275661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:13.047239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293613+0000","last_change":"2024-08-15T17:58:36.289433+0000","last_active":"2024-08-15T17:58:36.293613+0000","last_peered":"2024-08-15T17:58:36.293613+0000","last_clean":"2024-08-15T17:58:36.293613+0000","last_became_active":"2024-08-15T17:58:36.288625+0000","last_became_peered":"2024-08-15T17:58:36.288625+0000","last_unstale":"2024-08-15T17:58:36.293613+0000","last_undegraded":"2024-08-15T17:58:36.293613+0000","last_fullsized":"2024-08-15T17:58:36.293613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:47:03.102185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.273570+0000","last_change":"2024-08-15T17:58:35.276117+0000","last_active":"2024-08-15T17:58:36.273570+0000","last_peered":"2024-08-15T17:58:36.273570+0000","last_clean":"2024-08-15T17:58:36.273570+0000","last_became_active":"2024-08-15T17:58:35.275707+0000","last_became_peered":"2024-08-15T17:58:35.275707+0000","last_unstale":"2024-08-15T17:58:36.273570+0000","last_undegraded":"2024-08-15T17:58:36.273570+0000","last_fullsized":"2024-08-15T17:58:36.273570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:40:09.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293636+0000","last_change":"2024-08-15T17:58:36.290279+0000","last_active":"2024-08-15T17:58:36.293636+0000","last_peered":"2024-08-15T17:58:36.293636+0000","last_clean":"2024-08-15T17:58:36.293636+0000","last_became_active":"2024-08-15T17:58:36.289946+0000","last_became_peered":"2024-08-15T17:58:36.289946+0000","last_unstale":"2024-08-15T17:58:36.293636+0000","last_undegraded":"2024-08-15T17:58:36.293636+0000","last_fullsized":"2024-08-15T17:58:36.293636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:10:59.046094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298629+0000","last_change":"2024-08-15T17:58:35.274655+0000","last_active":"2024-08-15T17:58:36.298629+0000","last_peered":"2024-08-15T17:58:36.298629+0000","last_clean":"2024-08-15T17:58:36.298629+0000","last_became_active":"2024-08-15T17:58:35.273860+0000","last_became_peered":"2024-08-15T17:58:35.273860+0000","last_unstale":"2024-08-15T17:58:36.298629+0000","last_undegraded":"2024-08-15T17:58:36.298629+0000","last_fullsized":"2024-08-15T17:58:36.298629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:32:57.548918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.294347+0000","last_change":"2024-08-15T17:58:36.285943+0000","last_active":"2024-08-15T17:58:36.294347+0000","last_peered":"2024-08-15T17:58:36.294347+0000","last_clean":"2024-08-15T17:58:36.294347+0000","last_became_active":"2024-08-15T17:58:36.283605+0000","last_became_peered":"2024-08-15T17:58:36.283605+0000","last_unstale":"2024-08-15T17:58:36.294347+0000","last_undegraded":"2024-08-15T17:58:36.294347+0000","last_fullsized":"2024-08-15T17:58:36.294347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:07:28.572500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.282145+0000","last_change":"2024-08-15T17:58:36.271574+0000","last_active":"2024-08-15T17:58:36.282145+0000","last_peered":"2024-08-15T17:58:36.282145+0000","last_clean":"2024-08-15T17:58:36.282145+0000","last_became_active":"2024-08-15T17:58:36.271259+0000","last_became_peered":"2024-08-15T17:58:36.271259+0000","last_unstale":"2024-08-15T17:58:36.282145+0000","last_undegraded":"2024-08-15T17:58:36.282145+0000","last_fullsized":"2024-08-15T17:58:36.282145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:59.214672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298518+0000","last_change":"2024-08-15T17:58:35.277718+0000","last_active":"2024-08-15T17:58:36.298518+0000","last_peered":"2024-08-15T17:58:36.298518+0000","last_clean":"2024-08-15T17:58:36.298518+0000","last_became_active":"2024-08-15T17:58:35.277447+0000","last_became_peered":"2024-08-15T17:58:35.277447+0000","last_unstale":"2024-08-15T17:58:36.298518+0000","last_undegraded":"2024-08-15T17:58:36.298518+0000","last_fullsized":"2024-08-15T17:58:36.298518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:23:08.234362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288214+0000","last_change":"2024-08-15T17:58:36.281039+0000","last_active":"2024-08-15T17:58:36.288214+0000","last_peered":"2024-08-15T17:58:36.288214+0000","last_clean":"2024-08-15T17:58:36.288214+0000","last_became_active":"2024-08-15T17:58:36.280780+0000","last_became_peered":"2024-08-15T17:58:36.280780+0000","last_unstale":"2024-08-15T17:58:36.288214+0000","last_undegraded":"2024-08-15T17:58:36.288214+0000","last_fullsized":"2024-08-15T17:58:36.288214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:47:03.102185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.298615+0000","last_change":"2024-08-15T17:58:35.276674+0000","last_active":"2024-08-15T17:58:36.298615+0000","last_peered":"2024-08-15T17:58:36.298615+0000","last_clean":"2024-08-15T17:58:36.298615+0000","last_became_active":"2024-08-15T17:58:35.276363+0000","last_became_peered":"2024-08-15T17:58:35.276363+0000","last_unstale":"2024-08-15T17:58:36.298615+0000","last_undegraded":"2024-08-15T17:58:36.298615+0000","last_fullsized":"2024-08-15T17:58:36.298615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:20.838352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.288357+0000","last_change":"2024-08-15T17:58:36.282308+0000","last_active":"2024-08-15T17:58:36.288357+0000","last_peered":"2024-08-15T17:58:36.288357+0000","last_clean":"2024-08-15T17:58:36.288357+0000","last_became_active":"2024-08-15T17:58:36.281986+0000","last_became_peered":"2024-08-15T17:58:36.281986+0000","last_unstale":"2024-08-15T17:58:36.288357+0000","last_undegraded":"2024-08-15T17:58:36.288357+0000","last_fullsized":"2024-08-15T17:58:36.288357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:48:28.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.306258+0000","last_change":"2024-08-15T17:58:35.274614+0000","last_active":"2024-08-15T17:58:36.306258+0000","last_peered":"2024-08-15T17:58:36.306258+0000","last_clean":"2024-08-15T17:58:36.306258+0000","last_became_active":"2024-08-15T17:58:35.274185+0000","last_became_peered":"2024-08-15T17:58:35.274185+0000","last_unstale":"2024-08-15T17:58:36.306258+0000","last_undegraded":"2024-08-15T17:58:36.306258+0000","last_fullsized":"2024-08-15T17:58:36.306258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:41.642352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.293725+0000","last_change":"2024-08-15T17:58:36.290572+0000","last_active":"2024-08-15T17:58:36.293725+0000","last_peered":"2024-08-15T17:58:36.293725+0000","last_clean":"2024-08-15T17:58:36.293725+0000","last_became_active":"2024-08-15T17:58:36.290269+0000","last_became_peered":"2024-08-15T17:58:36.290269+0000","last_unstale":"2024-08-15T17:58:36.293725+0000","last_undegraded":"2024-08-15T17:58:36.293725+0000","last_fullsized":"2024-08-15T17:58:36.293725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:35.242412+0000","last_clean_scrub_stamp":"2024-08-15T17:58:35.242412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:56:49.069407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:58:36.301957+0000","last_change":"2024-08-15T17:58:35.276392+0000","last_active":"2024-08-15T17:58:36.301957+0000","last_peered":"2024-08-15T17:58:36.301957+0000","last_clean":"2024-08-15T17:58:36.301957+0000","last_became_active":"2024-08-15T17:58:35.276085+0000","last_became_peered":"2024-08-15T17:58:35.276085+0000","last_unstale":"2024-08-15T17:58:36.301957+0000","last_undegraded":"2024-08-15T17:58:36.301957+0000","last_fullsized":"2024-08-15T17:58:36.301957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:34.233433+0000","last_clean_scrub_stamp":"2024-08-15T17:58:34.233433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:10:36.097809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":806912,"data_stored":663739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]}]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":74353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":72927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.367}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":212992,"data_stored":72891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":74353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301724,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070116,"statfs":{"total":96636764160,"available":96327798784,"internally_reserved":0,"allocated":208896,"data_stored":72801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301728,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070112,"statfs":{"total":96636764160,"available":96327794688,"internally_reserved":0,"allocated":217088,"data_stored":72961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298160,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073680,"statfs":{"total":96636764160,"available":96331448320,"internally_reserved":0,"allocated":815104,"data_stored":663795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:58:44.272 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-15T17:58:44.272 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-15T17:58:44.272 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-08-15T17:58:44.273 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-08-15T17:58:44.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 Processor -- start 2024-08-15T17:58:44.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 -- start start 2024-08-15T17:58:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9e4c159040 0x7f9e4c0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9e4c15d5b0 0x7f9e4c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4c0cf190 con 0x7f9e4c0d0010 2024-08-15T17:58:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4c0d09a0 con 0x7f9e4c159040 2024-08-15T17:58:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0ce5e0 con 0x7f9e4c15d5b0 2024-08-15T17:58:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e50c71700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9e4c159040 0x7f9e4c0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e4bfff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9e4c0d0010 0x7f9e4c15cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40226/0 (socket says 172.21.15.38:40226) 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e4bfff700 1 -- 172.21.15.38:0/2560417144 learned_addr learned my addr 172.21.15.38:0/2560417144 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51472700 1 --2- 172.21.15.38:0/2560417144 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9e4c15d5b0 0x7f9e4c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e4b7fe700 1 -- 172.21.15.38:0/2560417144 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031111113 0 0) 0x7f9e4c0cf190 con 0x7f9e4c0d0010 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e4b7fe700 1 -- 172.21.15.38:0/2560417144 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e2c003570 con 0x7f9e4c0d0010 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e50c71700 1 -- 172.21.15.38:0/2560417144 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9e4c15d5b0 msgr2=0x7f9e4c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e50c71700 1 --2- 172.21.15.38:0/2560417144 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9e4c15d5b0 0x7f9e4c0d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e50c71700 1 -- 172.21.15.38:0/2560417144 >> v1:172.21.15.38:6789/0 conn(0x7f9e4c0d0010 legacy=0x7f9e4c15cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e50c71700 1 -- 172.21.15.38:0/2560417144 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c0ce860 con 0x7f9e4c159040 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e50c71700 1 --2- 172.21.15.38:0/2560417144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e4c159040 0x7f9e4c0cf620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0098e0 tx=0x7f9e3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a8d2ea86bd4bee server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e4b7fe700 1 -- 172.21.15.38:0/2560417144 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c011630 con 0x7f9e4c159040 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e4b7fe700 1 -- 172.21.15.38:0/2560417144 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c011790 con 0x7f9e4c159040 2024-08-15T17:58:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e4b7fe700 1 -- 172.21.15.38:0/2560417144 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c011b30 con 0x7f9e4c159040 2024-08-15T17:58:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 -- 172.21.15.38:0/2560417144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e4c159040 msgr2=0x7f9e4c0cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 --2- 172.21.15.38:0/2560417144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e4c159040 0x7f9e4c0cf620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0098e0 tx=0x7f9e3c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 -- 172.21.15.38:0/2560417144 shutdown_connections 2024-08-15T17:58:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 --2- 172.21.15.38:0/2560417144 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9e4c15d5b0 0x7f9e4c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 --2- 172.21.15.38:0/2560417144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e4c159040 0x7f9e4c0cf620 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 -- 172.21.15.38:0/2560417144 >> 172.21.15.38:0/2560417144 conn(0x7f9e4c068250 msgr2=0x7f9e4c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 -- 172.21.15.38:0/2560417144 shutdown_connections 2024-08-15T17:58:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.406+0000 7f9e51c73700 1 -- 172.21.15.38:0/2560417144 wait complete. 2024-08-15T17:58:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51c73700 1 Processor -- start 2024-08-15T17:58:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51c73700 1 -- start start 2024-08-15T17:58:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51c73700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9e4c0d0010 0x7f9e4c149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51c73700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4c15d5b0 0x7f9e4c0b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51c73700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e4c0b1310 0x7f9e4c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51c73700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0b24e0 con 0x7f9e4c0b1310 2024-08-15T17:58:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51c73700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0b2970 con 0x7f9e4c15d5b0 2024-08-15T17:58:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51c73700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c14a0d0 con 0x7f9e4c0d0010 2024-08-15T17:58:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51472700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e4c0b1310 0x7f9e4c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51472700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e4c0b1310 0x7f9e4c0b4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54004/0 (socket says 172.21.15.38:54004) 2024-08-15T17:58:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51472700 1 -- 172.21.15.38:0/2101404921 learned_addr learned my addr 172.21.15.38:0/2101404921 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e50c71700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9e4c0d0010 0x7f9e4c149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e4bfff700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4c15d5b0 0x7f9e4c0b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51472700 1 -- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9e4c0d0010 msgr2=0x7f9e4c149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51472700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9e4c0d0010 0x7f9e4c149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51472700 1 -- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4c15d5b0 msgr2=0x7f9e4c0b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51472700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4c15d5b0 0x7f9e4c0b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51472700 1 -- 172.21.15.38:0/2101404921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c009420 con 0x7f9e4c0b1310 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51472700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e4c0b1310 0x7f9e4c0b4f50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009df0 tx=0x7f9e4400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e48ff9700 1 -- 172.21.15.38:0/2101404921 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4400caa0 con 0x7f9e4c0b1310 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e48ff9700 1 -- 172.21.15.38:0/2101404921 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4400f040 con 0x7f9e4c0b1310 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51c73700 1 -- 172.21.15.38:0/2101404921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c0b18d0 con 0x7f9e4c0b1310 2024-08-15T17:58:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e48ff9700 1 -- 172.21.15.38:0/2101404921 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4400ce70 con 0x7f9e4c0b1310 2024-08-15T17:58:44.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.410+0000 7f9e51c73700 1 -- 172.21.15.38:0/2101404921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c0b1e90 con 0x7f9e4c0b1310 2024-08-15T17:58:44.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.414+0000 7f9e51c73700 1 -- 172.21.15.38:0/2101404921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4c0b2690 con 0x7f9e4c0b1310 2024-08-15T17:58:44.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.414+0000 7f9e48ff9700 1 -- 172.21.15.38:0/2101404921 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9e44014690 con 0x7f9e4c0b1310 2024-08-15T17:58:44.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.414+0000 7f9e48ff9700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9e30060860 0x7f9e30062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.414+0000 7f9e48ff9700 1 -- 172.21.15.38:0/2101404921 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9e4407d7a0 con 0x7f9e4c0b1310 2024-08-15T17:58:44.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.414+0000 7f9e50c71700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9e30060860 0x7f9e30062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.418+0000 7f9e50c71700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9e30060860 0x7f9e30062d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c067ea0 tx=0x7f9e3c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:44.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.422+0000 7f9e48ff9700 1 -- 172.21.15.38:0/2101404921 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9e4c0b2690 con 0x7f9e4c0b1310 2024-08-15T17:58:44.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 -- 172.21.15.38:0/2101404921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9e4c0643a0 con 0x7f9e4c0b1310 2024-08-15T17:58:44.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e48ff9700 1 -- 172.21.15.38:0/2101404921 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9e440515e0 con 0x7f9e4c0b1310 2024-08-15T17:58:44.615 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:44.616 INFO:teuthology.orchestra.run.smithi038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-15T17:58:44.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 -- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9e30060860 msgr2=0x7f9e30062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9e30060860 0x7f9e30062d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c067ea0 tx=0x7f9e3c017000 comp rx=0 tx=0).stop 2024-08-15T17:58:44.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 -- 172.21.15.38:0/2101404921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e4c0b1310 msgr2=0x7f9e4c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e4c0b1310 0x7f9e4c0b4f50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009df0 tx=0x7f9e4400b040 comp rx=0 tx=0).stop 2024-08-15T17:58:44.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 -- 172.21.15.38:0/2101404921 shutdown_connections 2024-08-15T17:58:44.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9e30060860 0x7f9e30062d10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e4c0b1310 0x7f9e4c0b4f50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4c15d5b0 0x7f9e4c0b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 --2- 172.21.15.38:0/2101404921 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9e4c0d0010 0x7f9e4c149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 -- 172.21.15.38:0/2101404921 >> 172.21.15.38:0/2101404921 conn(0x7f9e4c068250 msgr2=0x7f9e4c156990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:44.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 -- 172.21.15.38:0/2101404921 shutdown_connections 2024-08-15T17:58:44.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.614+0000 7f9e51c73700 1 -- 172.21.15.38:0/2101404921 wait complete. 2024-08-15T17:58:44.629 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-08-15T17:58:44.630 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:44.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.730+0000 7f4a9dfca700 1 Processor -- start 2024-08-15T17:58:44.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.730+0000 7f4a9dfca700 1 -- start start 2024-08-15T17:58:44.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9dfca700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4a980d0010 0x7f4a980d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9dfca700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a980cf190 con 0x7f4a98159040 2024-08-15T17:58:44.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9dfca700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a980d09a0 con 0x7f4a9815cea0 2024-08-15T17:58:44.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9dfca700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9815d9f0 con 0x7f4a980d0010 2024-08-15T17:58:44.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a97fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4a980d0010 0x7f4a980d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9cfc8700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4a98159040 0x7f4a980cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40234/0 (socket says 172.21.15.38:40234) 2024-08-15T17:58:44.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9cfc8700 1 -- 172.21.15.38:0/776975876 learned_addr learned my addr 172.21.15.38:0/776975876 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:44.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a97fff700 1 -- 172.21.15.38:0/776975876 >> v1:172.21.15.104:6789/0 conn(0x7f4a9815cea0 legacy=0x7f4a9815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a97fff700 1 -- 172.21.15.38:0/776975876 >> v1:172.21.15.38:6789/0 conn(0x7f4a98159040 legacy=0x7f4a980cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a977fe700 1 -- 172.21.15.38:0/776975876 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611658066 0 0) 0x7f4a980cf190 con 0x7f4a98159040 2024-08-15T17:58:44.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a97fff700 1 -- 172.21.15.38:0/776975876 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980ce5e0 con 0x7f4a980d0010 2024-08-15T17:58:44.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a97fff700 1 --2- 172.21.15.38:0/776975876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a980d0010 0x7f4a980d0460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0098e0 tx=0x7f4a8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74b720a372d71ce server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:44.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a977fe700 1 -- 172.21.15.38:0/776975876 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c009e80 con 0x7f4a980d0010 2024-08-15T17:58:44.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a977fe700 1 -- 172.21.15.38:0/776975876 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c00b860 con 0x7f4a980d0010 2024-08-15T17:58:44.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a977fe700 1 -- 172.21.15.38:0/776975876 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c00bbe0 con 0x7f4a980d0010 2024-08-15T17:58:44.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9dfca700 1 -- 172.21.15.38:0/776975876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a980d0010 msgr2=0x7f4a980d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9dfca700 1 --2- 172.21.15.38:0/776975876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a980d0010 0x7f4a980d0460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0098e0 tx=0x7f4a8c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9dfca700 1 -- 172.21.15.38:0/776975876 shutdown_connections 2024-08-15T17:58:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9dfca700 1 --2- 172.21.15.38:0/776975876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a980d0010 0x7f4a980d0460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9dfca700 1 -- 172.21.15.38:0/776975876 >> 172.21.15.38:0/776975876 conn(0x7f4a98068250 msgr2=0x7f4a98068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.734+0000 7f4a9dfca700 1 -- 172.21.15.38:0/776975876 shutdown_connections 2024-08-15T17:58:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9dfca700 1 -- 172.21.15.38:0/776975876 wait complete. 2024-08-15T17:58:44.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9dfca700 1 Processor -- start 2024-08-15T17:58:44.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9dfca700 1 -- start start 2024-08-15T17:58:44.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9dfca700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a980d0010 0x7f4a9816a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9dfca700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a98159040 0x7f4a980b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9dfca700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a9815cea0 0x7f4a98157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9dfca700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9816a610 con 0x7f4a9815cea0 2024-08-15T17:58:44.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9dfca700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9816dbc0 con 0x7f4a98159040 2024-08-15T17:58:44.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9dfca700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9816cfb0 con 0x7f4a980d0010 2024-08-15T17:58:44.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9d7c9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a9815cea0 0x7f4a98157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9d7c9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a9815cea0 0x7f4a98157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54010/0 (socket says 172.21.15.38:54010) 2024-08-15T17:58:44.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9d7c9700 1 -- 172.21.15.38:0/1168818829 learned_addr learned my addr 172.21.15.38:0/1168818829 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:44.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a97fff700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a98159040 0x7f4a980b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a9cfc8700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a980d0010 0x7f4a9816a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a97fff700 1 -- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a980d0010 msgr2=0x7f4a9816a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a97fff700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a980d0010 0x7f4a9816a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a97fff700 1 -- 172.21.15.38:0/1168818829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a9815cea0 msgr2=0x7f4a98157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a97fff700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a9815cea0 0x7f4a98157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a97fff700 1 -- 172.21.15.38:0/1168818829 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c009420 con 0x7f4a98159040 2024-08-15T17:58:44.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.738+0000 7f4a97fff700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a98159040 0x7f4a980b21f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a98070d70 tx=0x7f4a8c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.742+0000 7f4a94ff9700 1 -- 172.21.15.38:0/1168818829 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c01f070 con 0x7f4a98159040 2024-08-15T17:58:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.742+0000 7f4a94ff9700 1 -- 172.21.15.38:0/1168818829 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c000bc0 con 0x7f4a98159040 2024-08-15T17:58:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.742+0000 7f4a9dfca700 1 -- 172.21.15.38:0/1168818829 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9816d290 con 0x7f4a98159040 2024-08-15T17:58:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.742+0000 7f4a94ff9700 1 -- 172.21.15.38:0/1168818829 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c006af0 con 0x7f4a98159040 2024-08-15T17:58:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.742+0000 7f4a9dfca700 1 -- 172.21.15.38:0/1168818829 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9816d850 con 0x7f4a98159040 2024-08-15T17:58:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.742+0000 7f4a9dfca700 1 -- 172.21.15.38:0/1168818829 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a980643a0 con 0x7f4a98159040 2024-08-15T17:58:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.742+0000 7f4a94ff9700 1 -- 172.21.15.38:0/1168818829 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4a8c023070 con 0x7f4a98159040 2024-08-15T17:58:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.742+0000 7f4a94ff9700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4a84060890 0x7f4a84062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.742+0000 7f4a94ff9700 1 -- 172.21.15.38:0/1168818829 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4a8c07dd60 con 0x7f4a98159040 2024-08-15T17:58:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.742+0000 7f4a9cfc8700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4a84060890 0x7f4a84062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.746+0000 7f4a9cfc8700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4a84060890 0x7f4a84062d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4a980afad0 tx=0x7f4a88007f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:44.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.754+0000 7f4a94ff9700 1 -- 172.21.15.38:0/1168818829 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4a8c04e1c0 con 0x7f4a98159040 2024-08-15T17:58:44.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.922+0000 7f4a9dfca700 1 -- 172.21.15.38:0/1168818829 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a980ce010 con 0x7f4a98159040 2024-08-15T17:58:44.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.922+0000 7f4a94ff9700 1 -- 172.21.15.38:0/1168818829 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f4a8c02a090 con 0x7f4a98159040 2024-08-15T17:58:44.925 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:44.926 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 -- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4a84060890 msgr2=0x7f4a84062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4a84060890 0x7f4a84062d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4a980afad0 tx=0x7f4a88007f30 comp rx=0 tx=0).stop 2024-08-15T17:58:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 -- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a98159040 msgr2=0x7f4a980b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a98159040 0x7f4a980b21f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a98070d70 tx=0x7f4a8c017000 comp rx=0 tx=0).stop 2024-08-15T17:58:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 -- 172.21.15.38:0/1168818829 shutdown_connections 2024-08-15T17:58:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4a84060890 0x7f4a84062d40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a9815cea0 0x7f4a98157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a98159040 0x7f4a980b21f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 --2- 172.21.15.38:0/1168818829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a980d0010 0x7f4a9816a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 -- 172.21.15.38:0/1168818829 >> 172.21.15.38:0/1168818829 conn(0x7f4a98068250 msgr2=0x7f4a980af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 -- 172.21.15.38:0/1168818829 shutdown_connections 2024-08-15T17:58:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:44.926+0000 7f4a9dfca700 1 -- 172.21.15.38:0/1168818829 wait complete. 2024-08-15T17:58:44.940 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:58:34.079267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:58:34.621969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:58:44.940 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:58:45.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.050+0000 7fae9ce84700 1 Processor -- start 2024-08-15T17:58:45.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.050+0000 7fae9ce84700 1 -- start start 2024-08-15T17:58:45.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9ce84700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fae98069c90 0x7fae980b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9ce84700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae98069800 con 0x7fae9806a0e0 2024-08-15T17:58:45.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9ce84700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae980c89c0 con 0x7fae980cbc00 2024-08-15T17:58:45.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9ce84700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae980c8ca0 con 0x7fae98069c90 2024-08-15T17:58:45.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae95d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fae98069c90 0x7fae980b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae96d9d700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fae9806a0e0 0x7fae980cbfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40246/0 (socket says 172.21.15.38:40246) 2024-08-15T17:58:45.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae95d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fae98069c90 0x7fae980b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54026/0 (socket says 172.21.15.38:54026) 2024-08-15T17:58:45.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae96d9d700 1 -- 172.21.15.38:0/1366809145 learned_addr learned my addr 172.21.15.38:0/1366809145 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:45.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae95d9b700 1 -- 172.21.15.38:0/1366809145 >> v1:172.21.15.104:6789/0 conn(0x7fae980cbc00 legacy=0x7fae980ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae95d9b700 1 -- 172.21.15.38:0/1366809145 >> v1:172.21.15.38:6789/0 conn(0x7fae9806a0e0 legacy=0x7fae980cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae95d9b700 1 -- 172.21.15.38:0/1366809145 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae98069490 con 0x7fae98069c90 2024-08-15T17:58:45.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae95d9b700 1 --2- 172.21.15.38:0/1366809145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae98069c90 0x7fae980b0ed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae880098e0 tx=0x7fae880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b00690f83e3b784 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9559a700 1 -- 172.21.15.38:0/1366809145 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88009e80 con 0x7fae98069c90 2024-08-15T17:58:45.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9559a700 1 -- 172.21.15.38:0/1366809145 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8800b860 con 0x7fae98069c90 2024-08-15T17:58:45.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9559a700 1 -- 172.21.15.38:0/1366809145 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8800bbe0 con 0x7fae98069c90 2024-08-15T17:58:45.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9ce84700 1 -- 172.21.15.38:0/1366809145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae98069c90 msgr2=0x7fae980b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9ce84700 1 --2- 172.21.15.38:0/1366809145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae98069c90 0x7fae980b0ed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae880098e0 tx=0x7fae880091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9ce84700 1 -- 172.21.15.38:0/1366809145 shutdown_connections 2024-08-15T17:58:45.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9ce84700 1 --2- 172.21.15.38:0/1366809145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae98069c90 0x7fae980b0ed0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9ce84700 1 -- 172.21.15.38:0/1366809145 >> 172.21.15.38:0/1366809145 conn(0x7fae980caab0 msgr2=0x7fae980caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:45.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9ce84700 1 -- 172.21.15.38:0/1366809145 shutdown_connections 2024-08-15T17:58:45.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.054+0000 7fae9ce84700 1 -- 172.21.15.38:0/1366809145 wait complete. 2024-08-15T17:58:45.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae9ce84700 1 Processor -- start 2024-08-15T17:58:45.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae9ce84700 1 -- start start 2024-08-15T17:58:45.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae9ce84700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fae98069c90 0x7fae9816a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae9ce84700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae9806a0e0 0x7fae980c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae9ce84700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae980cbc00 0x7fae981668e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae9ce84700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9816a6c0 con 0x7fae980cbc00 2024-08-15T17:58:45.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae9ce84700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae980c7f20 con 0x7fae9806a0e0 2024-08-15T17:58:45.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae9ce84700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae980c2cd0 con 0x7fae98069c90 2024-08-15T17:58:45.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae96d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae980cbc00 0x7fae981668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae96d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae980cbc00 0x7fae981668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54030/0 (socket says 172.21.15.38:54030) 2024-08-15T17:58:45.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae96d9d700 1 -- 172.21.15.38:0/2405239502 learned_addr learned my addr 172.21.15.38:0/2405239502 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:45.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae9659c700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fae98069c90 0x7fae9816a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae95d9b700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae9806a0e0 0x7fae980c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae96d9d700 1 -- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fae98069c90 msgr2=0x7fae9816a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae96d9d700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fae98069c90 0x7fae9816a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae96d9d700 1 -- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae9806a0e0 msgr2=0x7fae980c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae96d9d700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae9806a0e0 0x7fae980c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae96d9d700 1 -- 172.21.15.38:0/2405239502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae88009420 con 0x7fae980cbc00 2024-08-15T17:58:45.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.058+0000 7fae95d9b700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae9806a0e0 0x7fae980c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:45.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.062+0000 7fae96d9d700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae980cbc00 0x7fae981668e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae9000cc40 tx=0x7fae90001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.062+0000 7fae8e7fc700 1 -- 172.21.15.38:0/2405239502 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9000a8b0 con 0x7fae980cbc00 2024-08-15T17:58:45.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.062+0000 7fae8e7fc700 1 -- 172.21.15.38:0/2405239502 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9000aa10 con 0x7fae980cbc00 2024-08-15T17:58:45.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.062+0000 7fae9ce84700 1 -- 172.21.15.38:0/2405239502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae980c8130 con 0x7fae980cbc00 2024-08-15T17:58:45.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.062+0000 7fae8e7fc700 1 -- 172.21.15.38:0/2405239502 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9000add0 con 0x7fae980cbc00 2024-08-15T17:58:45.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.062+0000 7fae9ce84700 1 -- 172.21.15.38:0/2405239502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae980c2960 con 0x7fae980cbc00 2024-08-15T17:58:45.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.062+0000 7fae9ce84700 1 -- 172.21.15.38:0/2405239502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae980c27c0 con 0x7fae980cbc00 2024-08-15T17:58:45.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.062+0000 7fae8e7fc700 1 -- 172.21.15.38:0/2405239502 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fae90007300 con 0x7fae980cbc00 2024-08-15T17:58:45.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.062+0000 7fae8e7fc700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fae78060860 0x7fae78062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.066+0000 7fae8e7fc700 1 -- 172.21.15.38:0/2405239502 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fae9007ce50 con 0x7fae980cbc00 2024-08-15T17:58:45.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.066+0000 7fae9659c700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fae78060860 0x7fae78062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.066+0000 7fae9659c700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fae78060860 0x7fae78062d10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae9806f050 tx=0x7fae80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.070+0000 7fae8e7fc700 1 -- 172.21.15.38:0/2405239502 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fae980c27c0 con 0x7fae980cbc00 2024-08-15T17:58:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.226+0000 7fae9ce84700 1 -- 172.21.15.38:0/2405239502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae980c27c0 con 0x7fae980cbc00 2024-08-15T17:58:45.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae8e7fc700 1 -- 172.21.15.38:0/2405239502 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae980c27c0 con 0x7fae980cbc00 2024-08-15T17:58:45.231 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T17:58:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 -- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fae78060860 msgr2=0x7fae78062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fae78060860 0x7fae78062d10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae9806f050 tx=0x7fae80009040 comp rx=0 tx=0).stop 2024-08-15T17:58:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 -- 172.21.15.38:0/2405239502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae980cbc00 msgr2=0x7fae981668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae980cbc00 0x7fae981668e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae9000cc40 tx=0x7fae90001a50 comp rx=0 tx=0).stop 2024-08-15T17:58:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 -- 172.21.15.38:0/2405239502 shutdown_connections 2024-08-15T17:58:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fae78060860 0x7fae78062d10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae980cbc00 0x7fae981668e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae9806a0e0 0x7fae980c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 --2- 172.21.15.38:0/2405239502 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fae98069c90 0x7fae9816a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 -- 172.21.15.38:0/2405239502 >> 172.21.15.38:0/2405239502 conn(0x7fae980caab0 msgr2=0x7fae980b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:45.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 -- 172.21.15.38:0/2405239502 shutdown_connections 2024-08-15T17:58:45.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.230+0000 7fae9ce84700 1 -- 172.21.15.38:0/2405239502 wait complete. 2024-08-15T17:58:45.245 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:58:45.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.354+0000 7f5e417fd700 1 Processor -- start 2024-08-15T17:58:45.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.354+0000 7f5e417fd700 1 -- start start 2024-08-15T17:58:45.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e3c0cfb00 0x7f5e3c067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3c158cc0 0x7f5e3c1590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c0cf670 con 0x7f5e3c067030 2024-08-15T17:58:45.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c0cea60 con 0x7f5e3c0cfb00 2024-08-15T17:58:45.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0ced40 con 0x7f5e3c158cc0 2024-08-15T17:58:45.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e337fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e3c0cfb00 0x7f5e3c067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e337fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e3c0cfb00 0x7f5e3c067400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54040/0 (socket says 172.21.15.38:54040) 2024-08-15T17:58:45.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e337fe700 1 -- 172.21.15.38:0/252454354 learned_addr learned my addr 172.21.15.38:0/252454354 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:45.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e40ffc700 1 --2- 172.21.15.38:0/252454354 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3c158cc0 0x7f5e3c1590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e337fe700 1 -- 172.21.15.38:0/252454354 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3c158cc0 msgr2=0x7f5e3c1590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:45.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e337fe700 1 --2- 172.21.15.38:0/252454354 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3c158cc0 0x7f5e3c1590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e337fe700 1 -- 172.21.15.38:0/252454354 >> v1:172.21.15.104:6789/0 conn(0x7f5e3c067030 legacy=0x7f5e3c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e337fe700 1 -- 172.21.15.38:0/252454354 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c0cf020 con 0x7f5e3c0cfb00 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e337fe700 1 --2- 172.21.15.38:0/252454354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c0cfb00 0x7f5e3c067400 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e240099b0 tx=0x7f5e24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1ef8a2303277ba server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e3b7fe700 1 -- 172.21.15.38:0/252454354 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24009ea0 con 0x7f5e3c0cfb00 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e3b7fe700 1 -- 172.21.15.38:0/252454354 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2400b860 con 0x7f5e3c0cfb00 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e3b7fe700 1 -- 172.21.15.38:0/252454354 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400bc00 con 0x7f5e3c0cfb00 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 -- 172.21.15.38:0/252454354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c0cfb00 msgr2=0x7f5e3c067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 --2- 172.21.15.38:0/252454354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c0cfb00 0x7f5e3c067400 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e240099b0 tx=0x7f5e24009250 comp rx=0 tx=0).stop 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 -- 172.21.15.38:0/252454354 shutdown_connections 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 --2- 172.21.15.38:0/252454354 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3c158cc0 0x7f5e3c1590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 --2- 172.21.15.38:0/252454354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c0cfb00 0x7f5e3c067400 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 -- 172.21.15.38:0/252454354 >> 172.21.15.38:0/252454354 conn(0x7f5e3c0d0b50 msgr2=0x7f5e3c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 -- 172.21.15.38:0/252454354 shutdown_connections 2024-08-15T17:58:45.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.358+0000 7f5e417fd700 1 -- 172.21.15.38:0/252454354 wait complete. 2024-08-15T17:58:45.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e417fd700 1 Processor -- start 2024-08-15T17:58:45.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e417fd700 1 -- start start 2024-08-15T17:58:45.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e417fd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e3c067030 0x7f5e3c143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e417fd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3c0cfb00 0x7f5e3c0b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e417fd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c158cc0 0x7f5e3c0b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e417fd700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c15a790 con 0x7f5e3c158cc0 2024-08-15T17:58:45.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e417fd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c159b80 con 0x7f5e3c067030 2024-08-15T17:58:45.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e417fd700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c159e60 con 0x7f5e3c0cfb00 2024-08-15T17:58:45.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e40ffc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c158cc0 0x7f5e3c0b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e337fe700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3c0cfb00 0x7f5e3c0b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e3bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e3c067030 0x7f5e3c143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e40ffc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c158cc0 0x7f5e3c0b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54050/0 (socket says 172.21.15.38:54050) 2024-08-15T17:58:45.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e337fe700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3c0cfb00 0x7f5e3c0b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:44352/0 (socket says 172.21.15.38:44352) 2024-08-15T17:58:45.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e40ffc700 1 -- 172.21.15.38:0/3732549183 learned_addr learned my addr 172.21.15.38:0/3732549183 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:45.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e3bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e3c067030 0x7f5e3c143a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:36198/0 (socket says 172.21.15.38:36198) 2024-08-15T17:58:45.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e40ffc700 1 -- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3c0cfb00 msgr2=0x7f5e3c0b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e40ffc700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3c0cfb00 0x7f5e3c0b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e40ffc700 1 -- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e3c067030 msgr2=0x7f5e3c143a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e40ffc700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e3c067030 0x7f5e3c143a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e40ffc700 1 -- 172.21.15.38:0/3732549183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240094f0 con 0x7f5e3c158cc0 2024-08-15T17:58:45.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.362+0000 7f5e40ffc700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c158cc0 0x7f5e3c0b5250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009df0 tx=0x7f5e3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.366+0000 7f5e38ff9700 1 -- 172.21.15.38:0/3732549183 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400caa0 con 0x7f5e3c158cc0 2024-08-15T17:58:45.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.366+0000 7f5e38ff9700 1 -- 172.21.15.38:0/3732549183 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3400f040 con 0x7f5e3c158cc0 2024-08-15T17:58:45.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.366+0000 7f5e417fd700 1 -- 172.21.15.38:0/3732549183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c15a140 con 0x7f5e3c158cc0 2024-08-15T17:58:45.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.366+0000 7f5e38ff9700 1 -- 172.21.15.38:0/3732549183 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400ce70 con 0x7f5e3c158cc0 2024-08-15T17:58:45.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.366+0000 7f5e417fd700 1 -- 172.21.15.38:0/3732549183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c143fd0 con 0x7f5e3c158cc0 2024-08-15T17:58:45.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.366+0000 7f5e417fd700 1 -- 172.21.15.38:0/3732549183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c15a900 con 0x7f5e3c158cc0 2024-08-15T17:58:45.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.366+0000 7f5e38ff9700 1 -- 172.21.15.38:0/3732549183 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5e34014690 con 0x7f5e3c158cc0 2024-08-15T17:58:45.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.366+0000 7f5e38ff9700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e1c060860 0x7f5e1c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.366+0000 7f5e38ff9700 1 -- 172.21.15.38:0/3732549183 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e3407d440 con 0x7f5e3c158cc0 2024-08-15T17:58:45.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.366+0000 7f5e3bfff700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e1c060860 0x7f5e1c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.370+0000 7f5e3bfff700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e1c060860 0x7f5e1c062d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c003a80 tx=0x7f5e2c007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.374+0000 7f5e38ff9700 1 -- 172.21.15.38:0/3732549183 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5e34051320 con 0x7f5e3c158cc0 2024-08-15T17:58:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 -- 172.21.15.38:0/3732549183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e3c0638d0 con 0x7f5e3c158cc0 2024-08-15T17:58:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e38ff9700 1 -- 172.21.15.38:0/3732549183 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e34050eb0 con 0x7f5e3c158cc0 2024-08-15T17:58:45.535 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T17:58:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 -- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e1c060860 msgr2=0x7f5e1c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e1c060860 0x7f5e1c062d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c003a80 tx=0x7f5e2c007ec0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 -- 172.21.15.38:0/3732549183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c158cc0 msgr2=0x7f5e3c0b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c158cc0 0x7f5e3c0b5250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009df0 tx=0x7f5e3400b040 comp rx=0 tx=0).stop 2024-08-15T17:58:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 -- 172.21.15.38:0/3732549183 shutdown_connections 2024-08-15T17:58:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e1c060860 0x7f5e1c062d10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c158cc0 0x7f5e3c0b5250 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3c0cfb00 0x7f5e3c0b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 --2- 172.21.15.38:0/3732549183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e3c067030 0x7f5e3c143a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 -- 172.21.15.38:0/3732549183 >> 172.21.15.38:0/3732549183 conn(0x7f5e3c0d0b50 msgr2=0x7f5e3c0ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:45.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 -- 172.21.15.38:0/3732549183 shutdown_connections 2024-08-15T17:58:45.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.534+0000 7f5e417fd700 1 -- 172.21.15.38:0/3732549183 wait complete. 2024-08-15T17:58:45.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:58:45.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.654+0000 7f4f53fff700 1 Processor -- start 2024-08-15T17:58:45.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.654+0000 7f4f53fff700 1 -- start start 2024-08-15T17:58:45.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f53fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f541552a0 0x7f4f540b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f53fff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f540ae490 con 0x7f4f540b2370 2024-08-15T17:58:45.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f53fff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f54156d90 con 0x7f4f54068db0 2024-08-15T17:58:45.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f53fff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f540ae920 con 0x7f4f541552a0 2024-08-15T17:58:45.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f52ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f541552a0 0x7f4f540b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f52ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f541552a0 0x7f4f540b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54062/0 (socket says 172.21.15.38:54062) 2024-08-15T17:58:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f527fc700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4f540b2370 0x7f4f540686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40254/0 (socket says 172.21.15.38:40254) 2024-08-15T17:58:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f52ffd700 1 -- 172.21.15.38:0/3547476248 learned_addr learned my addr 172.21.15.38:0/3547476248 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f52ffd700 1 -- 172.21.15.38:0/3547476248 >> v1:172.21.15.104:6789/0 conn(0x7f4f54068db0 legacy=0x7f4f54155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f52ffd700 1 -- 172.21.15.38:0/3547476248 >> v1:172.21.15.38:6789/0 conn(0x7f4f540b2370 legacy=0x7f4f540686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f52ffd700 1 -- 172.21.15.38:0/3547476248 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f540ad880 con 0x7f4f541552a0 2024-08-15T17:58:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f52ffd700 1 --2- 172.21.15.38:0/3547476248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f541552a0 0x7f4f540b1e30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400bc50 tx=0x7f4f4400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8217029d613bacc server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f51ffb700 1 -- 172.21.15.38:0/3547476248 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400be60 con 0x7f4f541552a0 2024-08-15T17:58:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f51ffb700 1 -- 172.21.15.38:0/3547476248 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4400de00 con 0x7f4f541552a0 2024-08-15T17:58:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.658+0000 7f4f51ffb700 1 -- 172.21.15.38:0/3547476248 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440196b0 con 0x7f4f541552a0 2024-08-15T17:58:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 -- 172.21.15.38:0/3547476248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f541552a0 msgr2=0x7f4f540b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 --2- 172.21.15.38:0/3547476248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f541552a0 0x7f4f540b1e30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400bc50 tx=0x7f4f4400b4f0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 -- 172.21.15.38:0/3547476248 shutdown_connections 2024-08-15T17:58:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 --2- 172.21.15.38:0/3547476248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f541552a0 0x7f4f540b1e30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 -- 172.21.15.38:0/3547476248 >> 172.21.15.38:0/3547476248 conn(0x7f4f5406e350 msgr2=0x7f4f5406b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 -- 172.21.15.38:0/3547476248 shutdown_connections 2024-08-15T17:58:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 -- 172.21.15.38:0/3547476248 wait complete. 2024-08-15T17:58:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 Processor -- start 2024-08-15T17:58:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 -- start start 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f54068db0 0x7f4f540a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f540b2370 0x7f4f54142e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f541552a0 0x7f4f54150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f540a6d90 con 0x7f4f54068db0 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f540a5ea0 con 0x7f4f540b2370 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.662+0000 7f4f53fff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f540a6180 con 0x7f4f541552a0 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f52ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f54068db0 0x7f4f540a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f52ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f54068db0 0x7f4f540a9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54072/0 (socket says 172.21.15.38:54072) 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f52ffd700 1 -- 172.21.15.38:0/3850898561 learned_addr learned my addr 172.21.15.38:0/3850898561 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f527fc700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f540b2370 0x7f4f54142e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f537fe700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f541552a0 0x7f4f54150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f52ffd700 1 -- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f541552a0 msgr2=0x7f4f54150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f52ffd700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f541552a0 0x7f4f54150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f52ffd700 1 -- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f540b2370 msgr2=0x7f4f54142e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f52ffd700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f540b2370 0x7f4f54142e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f52ffd700 1 -- 172.21.15.38:0/3850898561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4400b790 con 0x7f4f54068db0 2024-08-15T17:58:45.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f52ffd700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f54068db0 0x7f4f540a9330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4f54064c90 tx=0x7f4f4401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f3affd700 1 -- 172.21.15.38:0/3850898561 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44010540 con 0x7f4f54068db0 2024-08-15T17:58:45.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f3affd700 1 -- 172.21.15.38:0/3850898561 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f440106a0 con 0x7f4f54068db0 2024-08-15T17:58:45.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f53fff700 1 -- 172.21.15.38:0/3850898561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f540a6460 con 0x7f4f54068db0 2024-08-15T17:58:45.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f3affd700 1 -- 172.21.15.38:0/3850898561 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44010a40 con 0x7f4f54068db0 2024-08-15T17:58:45.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f53fff700 1 -- 172.21.15.38:0/3850898561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f540a6a20 con 0x7f4f54068db0 2024-08-15T17:58:45.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f53fff700 1 -- 172.21.15.38:0/3850898561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f540638f0 con 0x7f4f54068db0 2024-08-15T17:58:45.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.666+0000 7f4f3affd700 1 -- 172.21.15.38:0/3850898561 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4f44022030 con 0x7f4f54068db0 2024-08-15T17:58:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.670+0000 7f4f3affd700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4f40060860 0x7f4f40062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.670+0000 7f4f3affd700 1 -- 172.21.15.38:0/3850898561 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4f44014070 con 0x7f4f54068db0 2024-08-15T17:58:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.670+0000 7f4f527fc700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4f40060860 0x7f4f40062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.670+0000 7f4f527fc700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4f40060860 0x7f4f40062d10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f5414c080 tx=0x7f4f48007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.678+0000 7f4f3affd700 1 -- 172.21.15.38:0/3850898561 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4f440511e0 con 0x7f4f54068db0 2024-08-15T17:58:45.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.842+0000 7f4f53fff700 1 -- 172.21.15.38:0/3850898561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4f540ad2f0 con 0x7f4f54068db0 2024-08-15T17:58:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.842+0000 7f4f3affd700 1 -- 172.21.15.38:0/3850898561 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f44023380 con 0x7f4f54068db0 2024-08-15T17:58:45.845 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-08-15T17:58:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 -- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4f40060860 msgr2=0x7f4f40062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4f40060860 0x7f4f40062d10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f5414c080 tx=0x7f4f48007eb0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 -- 172.21.15.38:0/3850898561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f54068db0 msgr2=0x7f4f540a9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f54068db0 0x7f4f540a9330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4f54064c90 tx=0x7f4f4401b000 comp rx=0 tx=0).stop 2024-08-15T17:58:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 -- 172.21.15.38:0/3850898561 shutdown_connections 2024-08-15T17:58:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4f40060860 0x7f4f40062d10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f541552a0 0x7f4f54150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f540b2370 0x7f4f54142e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 --2- 172.21.15.38:0/3850898561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f54068db0 0x7f4f540a9330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 -- 172.21.15.38:0/3850898561 >> 172.21.15.38:0/3850898561 conn(0x7f4f5406e350 msgr2=0x7f4f540a5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:45.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 -- 172.21.15.38:0/3850898561 shutdown_connections 2024-08-15T17:58:45.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.846+0000 7f4f53fff700 1 -- 172.21.15.38:0/3850898561 wait complete. 2024-08-15T17:58:45.860 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:58:45.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.966+0000 7f5e3d897700 1 Processor -- start 2024-08-15T17:58:45.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.966+0000 7f5e3d897700 1 -- start start 2024-08-15T17:58:45.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.966+0000 7f5e3d897700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5e380d0010 0x7f5e380d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.966+0000 7f5e3d897700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3815cea0 0x7f5e3815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.966+0000 7f5e3d897700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e380cf190 con 0x7f5e38159040 2024-08-15T17:58:45.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.966+0000 7f5e3d897700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e380d09a0 con 0x7f5e380d0010 2024-08-15T17:58:45.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.966+0000 7f5e3d897700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380ce5e0 con 0x7f5e3815cea0 2024-08-15T17:58:45.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3c895700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5e38159040 0x7f5e380cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40262/0 (socket says 172.21.15.38:40262) 2024-08-15T17:58:45.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3c895700 1 -- 172.21.15.38:0/3595203542 learned_addr learned my addr 172.21.15.38:0/3595203542 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:45.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d096700 1 --2- 172.21.15.38:0/3595203542 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3815cea0 0x7f5e3815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e37fff700 1 --2- 172.21.15.38:0/3595203542 >> v2:172.21.15.104:3300/0 conn(0x7f5e380d0010 0x7f5e380d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e377fe700 1 -- 172.21.15.38:0/3595203542 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442459010 0 0) 0x7f5e380cf190 con 0x7f5e38159040 2024-08-15T17:58:45.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e377fe700 1 -- 172.21.15.38:0/3595203542 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e20003570 con 0x7f5e38159040 2024-08-15T17:58:45.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d096700 1 -- 172.21.15.38:0/3595203542 >> v2:172.21.15.104:3300/0 conn(0x7f5e380d0010 msgr2=0x7f5e380d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:45.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d096700 1 --2- 172.21.15.38:0/3595203542 >> v2:172.21.15.104:3300/0 conn(0x7f5e380d0010 0x7f5e380d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d096700 1 -- 172.21.15.38:0/3595203542 >> v1:172.21.15.38:6789/0 conn(0x7f5e38159040 legacy=0x7f5e380cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d096700 1 -- 172.21.15.38:0/3595203542 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e380ce860 con 0x7f5e3815cea0 2024-08-15T17:58:45.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d096700 1 --2- 172.21.15.38:0/3595203542 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3815cea0 0x7f5e3815d320 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009e60 tx=0x7f5e30009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7b00f3890b10907 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e377fe700 1 -- 172.21.15.38:0/3595203542 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3000e070 con 0x7f5e3815cea0 2024-08-15T17:58:45.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e377fe700 1 -- 172.21.15.38:0/3595203542 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3000bab0 con 0x7f5e3815cea0 2024-08-15T17:58:45.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e377fe700 1 -- 172.21.15.38:0/3595203542 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3000be50 con 0x7f5e3815cea0 2024-08-15T17:58:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d897700 1 -- 172.21.15.38:0/3595203542 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3815cea0 msgr2=0x7f5e3815d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d897700 1 --2- 172.21.15.38:0/3595203542 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3815cea0 0x7f5e3815d320 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009e60 tx=0x7f5e30009780 comp rx=0 tx=0).stop 2024-08-15T17:58:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d897700 1 -- 172.21.15.38:0/3595203542 shutdown_connections 2024-08-15T17:58:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d897700 1 --2- 172.21.15.38:0/3595203542 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e3815cea0 0x7f5e3815d320 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d897700 1 --2- 172.21.15.38:0/3595203542 >> v2:172.21.15.104:3300/0 conn(0x7f5e380d0010 0x7f5e380d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d897700 1 -- 172.21.15.38:0/3595203542 >> 172.21.15.38:0/3595203542 conn(0x7f5e38068250 msgr2=0x7f5e38068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d897700 1 -- 172.21.15.38:0/3595203542 shutdown_connections 2024-08-15T17:58:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.970+0000 7f5e3d897700 1 -- 172.21.15.38:0/3595203542 wait complete. 2024-08-15T17:58:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3d897700 1 Processor -- start 2024-08-15T17:58:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3d897700 1 -- start start 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3d897700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e380d0010 0x7f5e38150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3d897700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e38159040 0x7f5e38169530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3d897700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e3815cea0 0x7f5e3815b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3d897700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3815afa0 con 0x7f5e38159040 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3d897700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e381510d0 con 0x7f5e3815cea0 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3d897700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380aff20 con 0x7f5e380d0010 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e37fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e38159040 0x7f5e38169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e37fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e38159040 0x7f5e38169530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54074/0 (socket says 172.21.15.38:54074) 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e37fff700 1 -- 172.21.15.38:0/3575113815 learned_addr learned my addr 172.21.15.38:0/3575113815 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3d096700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e3815cea0 0x7f5e3815b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3c895700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e380d0010 0x7f5e38150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e37fff700 1 -- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e380d0010 msgr2=0x7f5e38150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e37fff700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e380d0010 0x7f5e38150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e37fff700 1 -- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e3815cea0 msgr2=0x7f5e3815b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:45.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e37fff700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e3815cea0 0x7f5e3815b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:45.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e37fff700 1 -- 172.21.15.38:0/3575113815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e30009a20 con 0x7f5e38159040 2024-08-15T17:58:45.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e37fff700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e38159040 0x7f5e38169530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00e930 tx=0x7f5e2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e34ff9700 1 -- 172.21.15.38:0/3575113815 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00eed0 con 0x7f5e38159040 2024-08-15T17:58:45.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e34ff9700 1 -- 172.21.15.38:0/3575113815 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c00c850 con 0x7f5e38159040 2024-08-15T17:58:45.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3d897700 1 -- 172.21.15.38:0/3575113815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e380b0200 con 0x7f5e38159040 2024-08-15T17:58:45.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e34ff9700 1 -- 172.21.15.38:0/3575113815 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00cbf0 con 0x7f5e38159040 2024-08-15T17:58:45.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.974+0000 7f5e3d897700 1 -- 172.21.15.38:0/3575113815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e380b07c0 con 0x7f5e38159040 2024-08-15T17:58:45.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.978+0000 7f5e3d897700 1 -- 172.21.15.38:0/3575113815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3815b110 con 0x7f5e38159040 2024-08-15T17:58:45.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.978+0000 7f5e34ff9700 1 -- 172.21.15.38:0/3575113815 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5e2c013470 con 0x7f5e38159040 2024-08-15T17:58:45.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.978+0000 7f5e34ff9700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e24060890 0x7f5e24062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:45.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.978+0000 7f5e34ff9700 1 -- 172.21.15.38:0/3575113815 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e2c07ce20 con 0x7f5e38159040 2024-08-15T17:58:45.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.978+0000 7f5e3c895700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e24060890 0x7f5e24062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:45.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.978+0000 7f5e3c895700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e24060890 0x7f5e24062d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5e280058d0 tx=0x7f5e28009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:45.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:45.986+0000 7f5e34ff9700 1 -- 172.21.15.38:0/3575113815 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5e2c04d1d0 con 0x7f5e38159040 2024-08-15T17:58:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.182+0000 7f5e3d897700 1 -- 172.21.15.38:0/3575113815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e380643a0 con 0x7f5e38159040 2024-08-15T17:58:46.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e34ff9700 1 -- 172.21.15.38:0/3575113815 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f5e2c054e60 con 0x7f5e38159040 2024-08-15T17:58:46.187 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:46.187 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2444719791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2444719791},{"type":"v1","addr":"172.21.15.38:6833","nonce":2444719791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2667173752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2667173752},{"type":"v1","addr":"172.21.15.38:6835","nonce":2667173752}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1185013072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1185013072},{"type":"v1","addr":"172.21.15.104:6835","nonce":1185013072}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:58:35.622492+0000","modified":"2024-08-15T17:58:38.885045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/3628618356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3628618356},{"type":"v1","addr":"172.21.15.104:6837","nonce":3628618356}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:58:46.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 -- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e24060890 msgr2=0x7f5e24062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e24060890 0x7f5e24062d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5e280058d0 tx=0x7f5e28009040 comp rx=0 tx=0).stop 2024-08-15T17:58:46.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 -- 172.21.15.38:0/3575113815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e38159040 msgr2=0x7f5e38169530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e38159040 0x7f5e38169530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00e930 tx=0x7f5e2c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 -- 172.21.15.38:0/3575113815 shutdown_connections 2024-08-15T17:58:46.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e24060890 0x7f5e24062d40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e3815cea0 0x7f5e3815b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e38159040 0x7f5e38169530 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 --2- 172.21.15.38:0/3575113815 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e380d0010 0x7f5e38150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 -- 172.21.15.38:0/3575113815 >> 172.21.15.38:0/3575113815 conn(0x7f5e38068250 msgr2=0x7f5e38157df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:46.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 -- 172.21.15.38:0/3575113815 shutdown_connections 2024-08-15T17:58:46.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.186+0000 7f5e3d897700 1 -- 172.21.15.38:0/3575113815 wait complete. 2024-08-15T17:58:46.191 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-08-15T17:58:46.201 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:46.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.302+0000 7fe2eaaed700 1 Processor -- start 2024-08-15T17:58:46.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.302+0000 7fe2eaaed700 1 -- start start 2024-08-15T17:58:46.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.302+0000 7fe2eaaed700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe2e40b14a0 0x7fe2e40cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.302+0000 7fe2eaaed700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe2e40cb390 0x7fe2e4160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.302+0000 7fe2eaaed700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e40cbaa0 con 0x7fe2e4154900 2024-08-15T17:58:46.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2eaaed700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e4068790 con 0x7fe2e40cb390 2024-08-15T17:58:46.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2eaaed700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e41583f0 con 0x7fe2e40b14a0 2024-08-15T17:58:46.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2e9aeb700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe2e4154900 0x7fe2e40b0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40264/0 (socket says 172.21.15.38:40264) 2024-08-15T17:58:46.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2e9aeb700 1 -- 172.21.15.38:0/2313155034 learned_addr learned my addr 172.21.15.38:0/2313155034 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:46.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2ea2ec700 1 --2- 172.21.15.38:0/2313155034 >> v2:172.21.15.104:3300/0 conn(0x7fe2e40cb390 0x7fe2e4160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2e92ea700 1 --2- 172.21.15.38:0/2313155034 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe2e40b14a0 0x7fe2e40cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2e8ae9700 1 -- 172.21.15.38:0/2313155034 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833833086 0 0) 0x7fe2e40cbaa0 con 0x7fe2e4154900 2024-08-15T17:58:46.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2e8ae9700 1 -- 172.21.15.38:0/2313155034 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2d0003570 con 0x7fe2e4154900 2024-08-15T17:58:46.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2ea2ec700 1 -- 172.21.15.38:0/2313155034 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe2e40b14a0 msgr2=0x7fe2e40cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:46.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2ea2ec700 1 --2- 172.21.15.38:0/2313155034 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe2e40b14a0 0x7fe2e40cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2ea2ec700 1 -- 172.21.15.38:0/2313155034 >> v1:172.21.15.38:6789/0 conn(0x7fe2e4154900 legacy=0x7fe2e40b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2ea2ec700 1 -- 172.21.15.38:0/2313155034 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4157630 con 0x7fe2e40cb390 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2ea2ec700 1 --2- 172.21.15.38:0/2313155034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2e40cb390 0x7fe2e4160ac0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009e60 tx=0x7fe2dc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=974f1d61b8ea0b14 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2e8ae9700 1 -- 172.21.15.38:0/2313155034 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00e070 con 0x7fe2e40cb390 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2e8ae9700 1 -- 172.21.15.38:0/2313155034 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc00bab0 con 0x7fe2e40cb390 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2e8ae9700 1 -- 172.21.15.38:0/2313155034 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00be70 con 0x7fe2e40cb390 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2eaaed700 1 -- 172.21.15.38:0/2313155034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2e40cb390 msgr2=0x7fe2e4160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2eaaed700 1 --2- 172.21.15.38:0/2313155034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2e40cb390 0x7fe2e4160ac0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009e60 tx=0x7fe2dc009780 comp rx=0 tx=0).stop 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2eaaed700 1 -- 172.21.15.38:0/2313155034 shutdown_connections 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2eaaed700 1 --2- 172.21.15.38:0/2313155034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2e40cb390 0x7fe2e4160ac0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2eaaed700 1 --2- 172.21.15.38:0/2313155034 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe2e40b14a0 0x7fe2e40cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2eaaed700 1 -- 172.21.15.38:0/2313155034 >> 172.21.15.38:0/2313155034 conn(0x7fe2e40678d0 msgr2=0x7fe2e4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:46.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2eaaed700 1 -- 172.21.15.38:0/2313155034 shutdown_connections 2024-08-15T17:58:46.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2eaaed700 1 -- 172.21.15.38:0/2313155034 wait complete. 2024-08-15T17:58:46.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.306+0000 7fe2eaaed700 1 Processor -- start 2024-08-15T17:58:46.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2eaaed700 1 -- start start 2024-08-15T17:58:46.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2eaaed700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2e40b14a0 0x7fe2e4141320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2eaaed700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe2e40cb390 0x7fe2e4151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2eaaed700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2e4154900 0x7fe2e4158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2eaaed700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e4141860 con 0x7fe2e4154900 2024-08-15T17:58:46.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2eaaed700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e415a4e0 con 0x7fe2e40b14a0 2024-08-15T17:58:46.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2eaaed700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e415b300 con 0x7fe2e40cb390 2024-08-15T17:58:46.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2ea2ec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2e4154900 0x7fe2e4158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2ea2ec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2e4154900 0x7fe2e4158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54088/0 (socket says 172.21.15.38:54088) 2024-08-15T17:58:46.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2ea2ec700 1 -- 172.21.15.38:0/470301300 learned_addr learned my addr 172.21.15.38:0/470301300 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2e92ea700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe2e40cb390 0x7fe2e4151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2e9aeb700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2e40b14a0 0x7fe2e4141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2ea2ec700 1 -- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe2e40cb390 msgr2=0x7fe2e4151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2ea2ec700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe2e40cb390 0x7fe2e4151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2ea2ec700 1 -- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2e40b14a0 msgr2=0x7fe2e4141320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2ea2ec700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2e40b14a0 0x7fe2e4141320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2ea2ec700 1 -- 172.21.15.38:0/470301300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc009a20 con 0x7fe2e4154900 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2ea2ec700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2e4154900 0x7fe2e4158530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2e406c2d0 tx=0x7fe2dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2d9ffb700 1 -- 172.21.15.38:0/470301300 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc012e10 con 0x7fe2e4154900 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2eaaed700 1 -- 172.21.15.38:0/470301300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e4151e80 con 0x7fe2e4154900 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2d9ffb700 1 -- 172.21.15.38:0/470301300 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc027470 con 0x7fe2e4154900 2024-08-15T17:58:46.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2d9ffb700 1 -- 172.21.15.38:0/470301300 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0277f0 con 0x7fe2e4154900 2024-08-15T17:58:46.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2eaaed700 1 -- 172.21.15.38:0/470301300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e415a9d0 con 0x7fe2e4154900 2024-08-15T17:58:46.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.310+0000 7fe2eaaed700 1 -- 172.21.15.38:0/470301300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e415b470 con 0x7fe2e4154900 2024-08-15T17:58:46.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.314+0000 7fe2d9ffb700 1 -- 172.21.15.38:0/470301300 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe2dc024070 con 0x7fe2e4154900 2024-08-15T17:58:46.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.314+0000 7fe2d9ffb700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe2c8060860 0x7fe2c8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.314+0000 7fe2d9ffb700 1 -- 172.21.15.38:0/470301300 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2dc07f270 con 0x7fe2e4154900 2024-08-15T17:58:46.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.314+0000 7fe2e9aeb700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe2c8060860 0x7fe2c8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.314+0000 7fe2e9aeb700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe2c8060860 0x7fe2c8062d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0007fd0 tx=0x7fe2e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:46.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.318+0000 7fe2d9ffb700 1 -- 172.21.15.38:0/470301300 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe2dc04f6c0 con 0x7fe2e4154900 2024-08-15T17:58:46.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.482+0000 7fe2eaaed700 1 -- 172.21.15.38:0/470301300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2e40643a0 con 0x7fe2e4154900 2024-08-15T17:58:46.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2d9ffb700 1 -- 172.21.15.38:0/470301300 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fe2dc052ce0 con 0x7fe2e4154900 2024-08-15T17:58:46.488 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:46.488 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:46.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2eaaed700 1 -- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe2c8060860 msgr2=0x7fe2c8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2eaaed700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe2c8060860 0x7fe2c8062d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0007fd0 tx=0x7fe2e0009040 comp rx=0 tx=0).stop 2024-08-15T17:58:46.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2eaaed700 1 -- 172.21.15.38:0/470301300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2e4154900 msgr2=0x7fe2e4158530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2eaaed700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2e4154900 0x7fe2e4158530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2e406c2d0 tx=0x7fe2dc019000 comp rx=0 tx=0).stop 2024-08-15T17:58:46.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2eaaed700 1 -- 172.21.15.38:0/470301300 shutdown_connections 2024-08-15T17:58:46.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2eaaed700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe2c8060860 0x7fe2c8062d10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2eaaed700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2e4154900 0x7fe2e4158530 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2eaaed700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe2e40cb390 0x7fe2e4151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2eaaed700 1 --2- 172.21.15.38:0/470301300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2e40b14a0 0x7fe2e4141320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2eaaed700 1 -- 172.21.15.38:0/470301300 >> 172.21.15.38:0/470301300 conn(0x7fe2e40678d0 msgr2=0x7fe2e4152710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:46.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.486+0000 7fe2eaaed700 1 -- 172.21.15.38:0/470301300 shutdown_connections 2024-08-15T17:58:46.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.490+0000 7fe2eaaed700 1 -- 172.21.15.38:0/470301300 wait complete. 2024-08-15T17:58:46.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:58:34.079267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:58:34.621969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:58:46.507 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:58:46.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.614+0000 7f53819a7700 1 Processor -- start 2024-08-15T17:58:46.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.614+0000 7f53819a7700 1 -- start start 2024-08-15T17:58:46.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.614+0000 7f53819a7700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f537c1596d0 0x7f537c066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.614+0000 7f53819a7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c15dbc0 0x7f537c067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.614+0000 7f53819a7700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537c0cba90 con 0x7f537c159300 2024-08-15T17:58:46.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.614+0000 7f53819a7700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537c0b2b60 con 0x7f537c1596d0 2024-08-15T17:58:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.614+0000 7f53819a7700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0b2940 con 0x7f537c15dbc0 2024-08-15T17:58:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f537a7fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f537c1596d0 0x7f537c066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f537affd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f537c159300 0x7f537c0b2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44156/0 (socket says 172.21.15.38:44156) 2024-08-15T17:58:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f537affd700 1 -- 172.21.15.38:0/444931951 learned_addr learned my addr 172.21.15.38:0/444931951 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f537b7fe700 1 --2- 172.21.15.38:0/444931951 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c15dbc0 0x7f537c067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f537a7fc700 1 -- 172.21.15.38:0/444931951 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c15dbc0 msgr2=0x7f537c067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f537a7fc700 1 --2- 172.21.15.38:0/444931951 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c15dbc0 0x7f537c067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f537a7fc700 1 -- 172.21.15.38:0/444931951 >> v1:172.21.15.38:6789/0 conn(0x7f537c159300 legacy=0x7f537c0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f537a7fc700 1 -- 172.21.15.38:0/444931951 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0cb160 con 0x7f537c1596d0 2024-08-15T17:58:46.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f537a7fc700 1 --2- 172.21.15.38:0/444931951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f537c1596d0 0x7f537c066e00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f536c009910 tx=0x7f536c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88d61c791eb1ad8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f5379ffb700 1 -- 172.21.15.38:0/444931951 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c00b640 con 0x7f537c1596d0 2024-08-15T17:58:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f5379ffb700 1 -- 172.21.15.38:0/444931951 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c00b7a0 con 0x7f537c1596d0 2024-08-15T17:58:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f5379ffb700 1 -- 172.21.15.38:0/444931951 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c00bb60 con 0x7f537c1596d0 2024-08-15T17:58:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f53819a7700 1 -- 172.21.15.38:0/444931951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f537c1596d0 msgr2=0x7f537c066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f53819a7700 1 --2- 172.21.15.38:0/444931951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f537c1596d0 0x7f537c066e00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f536c009910 tx=0x7f536c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f53819a7700 1 -- 172.21.15.38:0/444931951 shutdown_connections 2024-08-15T17:58:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f53819a7700 1 --2- 172.21.15.38:0/444931951 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c15dbc0 0x7f537c067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f53819a7700 1 --2- 172.21.15.38:0/444931951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f537c1596d0 0x7f537c066e00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f53819a7700 1 -- 172.21.15.38:0/444931951 >> 172.21.15.38:0/444931951 conn(0x7f537c0682c0 msgr2=0x7f537c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f53819a7700 1 -- 172.21.15.38:0/444931951 shutdown_connections 2024-08-15T17:58:46.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.618+0000 7f53819a7700 1 -- 172.21.15.38:0/444931951 wait complete. 2024-08-15T17:58:46.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f53819a7700 1 Processor -- start 2024-08-15T17:58:46.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f53819a7700 1 -- start start 2024-08-15T17:58:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f53819a7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c159300 0x7f537c0c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f53819a7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c1596d0 0x7f537c0c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f53819a7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f537c15dbc0 0x7f537c0c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f53819a7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0c53d0 con 0x7f537c15dbc0 2024-08-15T17:58:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f53819a7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0c4c30 con 0x7f537c159300 2024-08-15T17:58:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f53819a7700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0c4950 con 0x7f537c1596d0 2024-08-15T17:58:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f537b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f537c15dbc0 0x7f537c0c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f537b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f537c15dbc0 0x7f537c0c3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34042/0 (socket says 172.21.15.38:34042) 2024-08-15T17:58:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f537b7fe700 1 -- 172.21.15.38:0/2239834613 learned_addr learned my addr 172.21.15.38:0/2239834613 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f537a7fc700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c1596d0 0x7f537c0c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f537affd700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c159300 0x7f537c0c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f537b7fe700 1 -- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c1596d0 msgr2=0x7f537c0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f537b7fe700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c1596d0 0x7f537c0c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f537b7fe700 1 -- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c159300 msgr2=0x7f537c0c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f537b7fe700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c159300 0x7f537c0c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f537b7fe700 1 -- 172.21.15.38:0/2239834613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c009420 con 0x7f537c15dbc0 2024-08-15T17:58:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f537b7fe700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f537c15dbc0 0x7f537c0c3cb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5374009e90 tx=0x7f537400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f5372ffd700 1 -- 172.21.15.38:0/2239834613 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537400cb70 con 0x7f537c15dbc0 2024-08-15T17:58:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f5372ffd700 1 -- 172.21.15.38:0/2239834613 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537400f040 con 0x7f537c15dbc0 2024-08-15T17:58:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f5372ffd700 1 -- 172.21.15.38:0/2239834613 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374014480 con 0x7f537c15dbc0 2024-08-15T17:58:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f53819a7700 1 -- 172.21.15.38:0/2239834613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c15c2a0 con 0x7f537c15dbc0 2024-08-15T17:58:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.622+0000 7f53819a7700 1 -- 172.21.15.38:0/2239834613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c15c860 con 0x7f537c15dbc0 2024-08-15T17:58:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.626+0000 7f53819a7700 1 -- 172.21.15.38:0/2239834613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c15c100 con 0x7f537c15dbc0 2024-08-15T17:58:46.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.626+0000 7f5372ffd700 1 -- 172.21.15.38:0/2239834613 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5374014830 con 0x7f537c15dbc0 2024-08-15T17:58:46.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.626+0000 7f5372ffd700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f535c060860 0x7f535c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.626+0000 7f5372ffd700 1 -- 172.21.15.38:0/2239834613 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f537407d530 con 0x7f537c15dbc0 2024-08-15T17:58:46.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.626+0000 7f537affd700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f535c060860 0x7f535c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.626+0000 7f537affd700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f535c060860 0x7f535c062d10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5364001950 tx=0x7f5364009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:46.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.634+0000 7f5372ffd700 1 -- 172.21.15.38:0/2239834613 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f537c15c100 con 0x7f537c15dbc0 2024-08-15T17:58:46.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.798+0000 7f53819a7700 1 -- 172.21.15.38:0/2239834613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f537c15c100 con 0x7f537c15dbc0 2024-08-15T17:58:46.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.798+0000 7f5372ffd700 1 -- 172.21.15.38:0/2239834613 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f537c15c100 con 0x7f537c15dbc0 2024-08-15T17:58:46.801 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T17:58:46.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 -- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f535c060860 msgr2=0x7f535c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f535c060860 0x7f535c062d10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5364001950 tx=0x7f5364009040 comp rx=0 tx=0).stop 2024-08-15T17:58:46.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 -- 172.21.15.38:0/2239834613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f537c15dbc0 msgr2=0x7f537c0c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f537c15dbc0 0x7f537c0c3cb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5374009e90 tx=0x7f537400b040 comp rx=0 tx=0).stop 2024-08-15T17:58:46.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 -- 172.21.15.38:0/2239834613 shutdown_connections 2024-08-15T17:58:46.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f535c060860 0x7f535c062d10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f537c15dbc0 0x7f537c0c3cb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c1596d0 0x7f537c0c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 --2- 172.21.15.38:0/2239834613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c159300 0x7f537c0c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 -- 172.21.15.38:0/2239834613 >> 172.21.15.38:0/2239834613 conn(0x7f537c0682c0 msgr2=0x7f537c1565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:46.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 -- 172.21.15.38:0/2239834613 shutdown_connections 2024-08-15T17:58:46.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.802+0000 7f53819a7700 1 -- 172.21.15.38:0/2239834613 wait complete. 2024-08-15T17:58:46.816 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:58:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.926+0000 7fcfaada6700 1 Processor -- start 2024-08-15T17:58:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.926+0000 7fcfaada6700 1 -- start start 2024-08-15T17:58:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfaada6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fcfa40b0e20 0x7fcfa4160e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfaada6700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa40af460 con 0x7fcfa4068db0 2024-08-15T17:58:46.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfaada6700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa40b02c0 con 0x7fcfa414cd50 2024-08-15T17:58:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfaada6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa40af6b0 con 0x7fcfa40b0e20 2024-08-15T17:58:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfa95a3700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcfa4068db0 0x7fcfa40b0710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44168/0 (socket says 172.21.15.38:44168) 2024-08-15T17:58:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfa95a3700 1 -- 172.21.15.38:0/218408459 learned_addr learned my addr 172.21.15.38:0/218408459 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfaa5a5700 1 --2- 172.21.15.38:0/218408459 >> v2:172.21.15.104:3300/0 conn(0x7fcfa40b0e20 0x7fcfa4160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfa8da2700 1 -- 172.21.15.38:0/218408459 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058198821 0 0) 0x7fcfa40af460 con 0x7fcfa4068db0 2024-08-15T17:58:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfa8da2700 1 -- 172.21.15.38:0/218408459 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf90003570 con 0x7fcfa4068db0 2024-08-15T17:58:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfaa5a5700 1 -- 172.21.15.38:0/218408459 >> v1:172.21.15.104:6789/0 conn(0x7fcfa414cd50 legacy=0x7fcfa40686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfaa5a5700 1 -- 172.21.15.38:0/218408459 >> v1:172.21.15.38:6789/0 conn(0x7fcfa4068db0 legacy=0x7fcfa40b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfa8da2700 1 -- 172.21.15.38:0/218408459 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 473183154 0 0) 0x7fcf90003570 con 0x7fcfa4068db0 2024-08-15T17:58:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfaa5a5700 1 -- 172.21.15.38:0/218408459 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa40af990 con 0x7fcfa40b0e20 2024-08-15T17:58:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfaa5a5700 1 --2- 172.21.15.38:0/218408459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa40b0e20 0x7fcfa4160e30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009e60 tx=0x7fcf9c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca585da1b58f6b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfa8da2700 1 -- 172.21.15.38:0/218408459 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c00e070 con 0x7fcfa40b0e20 2024-08-15T17:58:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfa8da2700 1 -- 172.21.15.38:0/218408459 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c00b850 con 0x7fcfa40b0e20 2024-08-15T17:58:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.930+0000 7fcfa8da2700 1 -- 172.21.15.38:0/218408459 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf940013f0 con 0x7fcfa40b0e20 2024-08-15T17:58:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 -- 172.21.15.38:0/218408459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa40b0e20 msgr2=0x7fcfa4160e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 --2- 172.21.15.38:0/218408459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa40b0e20 0x7fcfa4160e30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009e60 tx=0x7fcf9c009780 comp rx=0 tx=0).stop 2024-08-15T17:58:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 -- 172.21.15.38:0/218408459 shutdown_connections 2024-08-15T17:58:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 --2- 172.21.15.38:0/218408459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa40b0e20 0x7fcfa4160e30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 -- 172.21.15.38:0/218408459 >> 172.21.15.38:0/218408459 conn(0x7fcfa406d800 msgr2=0x7fcfa406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 -- 172.21.15.38:0/218408459 shutdown_connections 2024-08-15T17:58:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 -- 172.21.15.38:0/218408459 wait complete. 2024-08-15T17:58:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 Processor -- start 2024-08-15T17:58:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 -- start start 2024-08-15T17:58:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa4068db0 0x7fcfa406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa40b0e20 0x7fcfa4141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa414cd50 0x7fcfa40a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4159640 con 0x7fcfa40b0e20 2024-08-15T17:58:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa406b530 con 0x7fcfa4068db0 2024-08-15T17:58:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfaada6700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa415be10 con 0x7fcfa414cd50 2024-08-15T17:58:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfa95a3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa40b0e20 0x7fcfa4141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfa95a3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa40b0e20 0x7fcfa4141960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34054/0 (socket says 172.21.15.38:34054) 2024-08-15T17:58:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.934+0000 7fcfa95a3700 1 -- 172.21.15.38:0/2598340369 learned_addr learned my addr 172.21.15.38:0/2598340369 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfa9da4700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa4068db0 0x7fcfa406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfaa5a5700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa414cd50 0x7fcfa40a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfa95a3700 1 -- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa414cd50 msgr2=0x7fcfa40a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfa95a3700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa414cd50 0x7fcfa40a9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfa95a3700 1 -- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa4068db0 msgr2=0x7fcfa406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfa95a3700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa4068db0 0x7fcfa406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfa95a3700 1 -- 172.21.15.38:0/2598340369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c009a20 con 0x7fcfa40b0e20 2024-08-15T17:58:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfaa5a5700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa414cd50 0x7fcfa40a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfa95a3700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa40b0e20 0x7fcfa4141960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400cb90 tx=0x7fcf94001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcf99ffb700 1 -- 172.21.15.38:0/2598340369 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf94007150 con 0x7fcfa40b0e20 2024-08-15T17:58:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcf99ffb700 1 -- 172.21.15.38:0/2598340369 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf940072d0 con 0x7fcfa40b0e20 2024-08-15T17:58:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfaada6700 1 -- 172.21.15.38:0/2598340369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa4159080 con 0x7fcfa40b0e20 2024-08-15T17:58:46.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcf99ffb700 1 -- 172.21.15.38:0/2598340369 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf94007690 con 0x7fcfa40b0e20 2024-08-15T17:58:46.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfaada6700 1 -- 172.21.15.38:0/2598340369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa415baa0 con 0x7fcfa40b0e20 2024-08-15T17:58:46.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcfaada6700 1 -- 172.21.15.38:0/2598340369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa415b900 con 0x7fcfa40b0e20 2024-08-15T17:58:46.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.938+0000 7fcf99ffb700 1 -- 172.21.15.38:0/2598340369 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcf94004a40 con 0x7fcfa40b0e20 2024-08-15T17:58:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.942+0000 7fcf99ffb700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcf88060890 0x7fcf88062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.942+0000 7fcf99ffb700 1 -- 172.21.15.38:0/2598340369 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcf9407c3a0 con 0x7fcfa40b0e20 2024-08-15T17:58:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.942+0000 7fcfa9da4700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcf88060890 0x7fcf88062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:46.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.942+0000 7fcfa9da4700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcf88060890 0x7fcf88062d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0003b00 tx=0x7fcfa0007e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:46.950+0000 7fcf99ffb700 1 -- 172.21.15.38:0/2598340369 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcfa415b900 con 0x7fcfa40b0e20 2024-08-15T17:58:47.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 -- 172.21.15.38:0/2598340369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfa415b900 con 0x7fcfa40b0e20 2024-08-15T17:58:47.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcf99ffb700 1 -- 172.21.15.38:0/2598340369 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf94018020 con 0x7fcfa40b0e20 2024-08-15T17:58:47.120 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T17:58:47.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 -- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcf88060890 msgr2=0x7fcf88062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcf88060890 0x7fcf88062d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0003b00 tx=0x7fcfa0007e20 comp rx=0 tx=0).stop 2024-08-15T17:58:47.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 -- 172.21.15.38:0/2598340369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa40b0e20 msgr2=0x7fcfa4141960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa40b0e20 0x7fcfa4141960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400cb90 tx=0x7fcf94001a30 comp rx=0 tx=0).stop 2024-08-15T17:58:47.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 -- 172.21.15.38:0/2598340369 shutdown_connections 2024-08-15T17:58:47.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcf88060890 0x7fcf88062d40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa414cd50 0x7fcfa40a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa40b0e20 0x7fcfa4141960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 --2- 172.21.15.38:0/2598340369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa4068db0 0x7fcfa406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 -- 172.21.15.38:0/2598340369 >> 172.21.15.38:0/2598340369 conn(0x7fcfa406d800 msgr2=0x7fcfa4068300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:47.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 -- 172.21.15.38:0/2598340369 shutdown_connections 2024-08-15T17:58:47.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.118+0000 7fcfaada6700 1 -- 172.21.15.38:0/2598340369 wait complete. 2024-08-15T17:58:47.133 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:58:47.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.250+0000 7fc9e8f58700 1 Processor -- start 2024-08-15T17:58:47.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.250+0000 7fc9e8f58700 1 -- start start 2024-08-15T17:58:47.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9e40ab2e0 0x7fc9e40ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc9e40c9930 0x7fc9e40c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e40aae50 con 0x7fc9e40afd50 2024-08-15T17:58:47.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e40abc50 con 0x7fc9e40c9930 2024-08-15T17:58:47.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40ca2f0 con 0x7fc9e40ab2e0 2024-08-15T17:58:47.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc9e40c9930 0x7fc9e40c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e37fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc9e40afd50 0x7fc9e40a9860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44176/0 (socket says 172.21.15.38:44176) 2024-08-15T17:58:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e37fe700 1 -- 172.21.15.38:0/2854795967 learned_addr learned my addr 172.21.15.38:0/2854795967 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e2ffd700 1 --2- 172.21.15.38:0/2854795967 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9e40ab2e0 0x7fc9e40ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e3fff700 1 -- 172.21.15.38:0/2854795967 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9e40ab2e0 msgr2=0x7fc9e40ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e3fff700 1 --2- 172.21.15.38:0/2854795967 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9e40ab2e0 0x7fc9e40ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e3fff700 1 -- 172.21.15.38:0/2854795967 >> v1:172.21.15.38:6789/0 conn(0x7fc9e40afd50 legacy=0x7fc9e40a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e27fc700 1 -- 172.21.15.38:0/2854795967 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403565257 0 0) 0x7fc9e40aae50 con 0x7fc9e40afd50 2024-08-15T17:58:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e3fff700 1 -- 172.21.15.38:0/2854795967 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e40a9f70 con 0x7fc9e40c9930 2024-08-15T17:58:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e3fff700 1 --2- 172.21.15.38:0/2854795967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9e40c9930 0x7fc9e40c9db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009e60 tx=0x7fc9dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33966454ded23e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e27fc700 1 -- 172.21.15.38:0/2854795967 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00e070 con 0x7fc9e40c9930 2024-08-15T17:58:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e27fc700 1 -- 172.21.15.38:0/2854795967 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc00bb60 con 0x7fc9e40c9930 2024-08-15T17:58:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e27fc700 1 -- 172.21.15.38:0/2854795967 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc012430 con 0x7fc9e40c9930 2024-08-15T17:58:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 -- 172.21.15.38:0/2854795967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9e40c9930 msgr2=0x7fc9e40c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 --2- 172.21.15.38:0/2854795967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9e40c9930 0x7fc9e40c9db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009e60 tx=0x7fc9dc009780 comp rx=0 tx=0).stop 2024-08-15T17:58:47.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 -- 172.21.15.38:0/2854795967 shutdown_connections 2024-08-15T17:58:47.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 --2- 172.21.15.38:0/2854795967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9e40c9930 0x7fc9e40c9db0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 --2- 172.21.15.38:0/2854795967 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9e40ab2e0 0x7fc9e40ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 -- 172.21.15.38:0/2854795967 >> 172.21.15.38:0/2854795967 conn(0x7fc9e406e170 msgr2=0x7fc9e406e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:47.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.254+0000 7fc9e8f58700 1 -- 172.21.15.38:0/2854795967 shutdown_connections 2024-08-15T17:58:47.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e8f58700 1 -- 172.21.15.38:0/2854795967 wait complete. 2024-08-15T17:58:47.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e8f58700 1 Processor -- start 2024-08-15T17:58:47.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e8f58700 1 -- start start 2024-08-15T17:58:47.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e8f58700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9e40ab2e0 0x7fc9e4154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e8f58700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e40afd50 0x7fc9e4157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e8f58700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9e40c9930 0x7fc9e4157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e8f58700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40ae290 con 0x7fc9e40c9930 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e8f58700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4068960 con 0x7fc9e40afd50 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e8f58700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4068630 con 0x7fc9e40ab2e0 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9e40c9930 0x7fc9e4157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9e40c9930 0x7fc9e4157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34084/0 (socket says 172.21.15.38:34084) 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e3fff700 1 -- 172.21.15.38:0/3009212717 learned_addr learned my addr 172.21.15.38:0/3009212717 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e2ffd700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e40afd50 0x7fc9e4157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e37fe700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9e40ab2e0 0x7fc9e4154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e3fff700 1 -- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9e40ab2e0 msgr2=0x7fc9e4154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e3fff700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9e40ab2e0 0x7fc9e4154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e3fff700 1 -- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e40afd50 msgr2=0x7fc9e4157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e3fff700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e40afd50 0x7fc9e4157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e3fff700 1 -- 172.21.15.38:0/3009212717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc009a20 con 0x7fc9e40c9930 2024-08-15T17:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e2ffd700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e40afd50 0x7fc9e4157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e3fff700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9e40c9930 0x7fc9e4157fd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc012ad0 tx=0x7fc9dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9cb7fe700 1 -- 172.21.15.38:0/3009212717 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc012e50 con 0x7fc9e40c9930 2024-08-15T17:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9cb7fe700 1 -- 172.21.15.38:0/3009212717 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc01b600 con 0x7fc9e40c9930 2024-08-15T17:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e8f58700 1 -- 172.21.15.38:0/3009212717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e40ad9f0 con 0x7fc9e40c9930 2024-08-15T17:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9cb7fe700 1 -- 172.21.15.38:0/3009212717 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc01b9c0 con 0x7fc9e40c9930 2024-08-15T17:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.258+0000 7fc9e8f58700 1 -- 172.21.15.38:0/3009212717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e40adfb0 con 0x7fc9e40c9930 2024-08-15T17:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.262+0000 7fc9e8f58700 1 -- 172.21.15.38:0/3009212717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e4068af0 con 0x7fc9e40c9930 2024-08-15T17:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.262+0000 7fc9cb7fe700 1 -- 172.21.15.38:0/3009212717 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc9dc003050 con 0x7fc9e40c9930 2024-08-15T17:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.262+0000 7fc9cb7fe700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9d0060860 0x7fc9d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.262+0000 7fc9cb7fe700 1 -- 172.21.15.38:0/3009212717 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc9dc07ea50 con 0x7fc9e40c9930 2024-08-15T17:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.262+0000 7fc9e37fe700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9d0060860 0x7fc9d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.266+0000 7fc9e37fe700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9d0060860 0x7fc9d0062d10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40011e0 tx=0x7fc9d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:47.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.270+0000 7fc9cb7fe700 1 -- 172.21.15.38:0/3009212717 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc9dc04eea0 con 0x7fc9e40c9930 2024-08-15T17:58:47.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 -- 172.21.15.38:0/3009212717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9e40638f0 con 0x7fc9e40c9930 2024-08-15T17:58:47.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9cb7fe700 1 -- 172.21.15.38:0/3009212717 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc9dc0524c0 con 0x7fc9e40c9930 2024-08-15T17:58:47.439 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-08-15T17:58:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 -- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9d0060860 msgr2=0x7fc9d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9d0060860 0x7fc9d0062d10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40011e0 tx=0x7fc9d4009040 comp rx=0 tx=0).stop 2024-08-15T17:58:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 -- 172.21.15.38:0/3009212717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9e40c9930 msgr2=0x7fc9e4157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9e40c9930 0x7fc9e4157fd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc012ad0 tx=0x7fc9dc019000 comp rx=0 tx=0).stop 2024-08-15T17:58:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 -- 172.21.15.38:0/3009212717 shutdown_connections 2024-08-15T17:58:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9d0060860 0x7fc9d0062d10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9e40c9930 0x7fc9e4157fd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e40afd50 0x7fc9e4157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 --2- 172.21.15.38:0/3009212717 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9e40ab2e0 0x7fc9e4154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 -- 172.21.15.38:0/3009212717 >> 172.21.15.38:0/3009212717 conn(0x7fc9e406e170 msgr2=0x7fc9e406e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:47.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 -- 172.21.15.38:0/3009212717 shutdown_connections 2024-08-15T17:58:47.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.438+0000 7fc9e8f58700 1 -- 172.21.15.38:0/3009212717 wait complete. 2024-08-15T17:58:47.454 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:58:47.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc2a65b700 1 Processor -- start 2024-08-15T17:58:47.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc2a65b700 1 -- start start 2024-08-15T17:58:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc2a65b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdc24159040 0x7fdc240cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc2a65b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdc240d0010 0x7fdc240d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc2a65b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc240cf190 con 0x7fdc2415cea0 2024-08-15T17:58:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc2a65b700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2415d9f0 con 0x7fdc24159040 2024-08-15T17:58:47.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc2a65b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc240d09a0 con 0x7fdc240d0010 2024-08-15T17:58:47.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc29659700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdc24159040 0x7fdc240cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc29e5a700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdc2415cea0 0x7fdc2415d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44178/0 (socket says 172.21.15.38:44178) 2024-08-15T17:58:47.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc29e5a700 1 -- 172.21.15.38:0/2712934804 learned_addr learned my addr 172.21.15.38:0/2712934804 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:47.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc28e58700 1 --2- 172.21.15.38:0/2712934804 >> v2:172.21.15.104:3300/0 conn(0x7fdc240d0010 0x7fdc240d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc17fff700 1 -- 172.21.15.38:0/2712934804 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738372494 0 0) 0x7fdc240cf190 con 0x7fdc2415cea0 2024-08-15T17:58:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc17fff700 1 -- 172.21.15.38:0/2712934804 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc10003570 con 0x7fdc2415cea0 2024-08-15T17:58:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc29659700 1 -- 172.21.15.38:0/2712934804 >> v2:172.21.15.104:3300/0 conn(0x7fdc240d0010 msgr2=0x7fdc240d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc29659700 1 --2- 172.21.15.38:0/2712934804 >> v2:172.21.15.104:3300/0 conn(0x7fdc240d0010 0x7fdc240d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc29659700 1 -- 172.21.15.38:0/2712934804 >> v1:172.21.15.38:6789/0 conn(0x7fdc2415cea0 legacy=0x7fdc2415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc29659700 1 -- 172.21.15.38:0/2712934804 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240ce5e0 con 0x7fdc24159040 2024-08-15T17:58:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc29659700 1 --2- 172.21.15.38:0/2712934804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc24159040 0x7fdc240cf620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdc24067ef0 tx=0x7fdc200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cc6be9a8537c42 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc17fff700 1 -- 172.21.15.38:0/2712934804 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20011630 con 0x7fdc24159040 2024-08-15T17:58:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc17fff700 1 -- 172.21.15.38:0/2712934804 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc200117b0 con 0x7fdc24159040 2024-08-15T17:58:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc17fff700 1 -- 172.21.15.38:0/2712934804 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20011b70 con 0x7fdc24159040 2024-08-15T17:58:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc2a65b700 1 -- 172.21.15.38:0/2712934804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc24159040 msgr2=0x7fdc240cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.562+0000 7fdc2a65b700 1 --2- 172.21.15.38:0/2712934804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc24159040 0x7fdc240cf620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdc24067ef0 tx=0x7fdc200091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 -- 172.21.15.38:0/2712934804 shutdown_connections 2024-08-15T17:58:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 --2- 172.21.15.38:0/2712934804 >> v2:172.21.15.104:3300/0 conn(0x7fdc240d0010 0x7fdc240d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 --2- 172.21.15.38:0/2712934804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc24159040 0x7fdc240cf620 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 -- 172.21.15.38:0/2712934804 >> 172.21.15.38:0/2712934804 conn(0x7fdc24068250 msgr2=0x7fdc24068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 -- 172.21.15.38:0/2712934804 shutdown_connections 2024-08-15T17:58:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 -- 172.21.15.38:0/2712934804 wait complete. 2024-08-15T17:58:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 Processor -- start 2024-08-15T17:58:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 -- start start 2024-08-15T17:58:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc240d0010 0x7fdc240b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc24159040 0x7fdc2415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc2415cea0 0x7fdc24157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2415bec0 con 0x7fdc2415cea0 2024-08-15T17:58:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc240b1530 con 0x7fdc24159040 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc2a65b700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc240b1810 con 0x7fdc240d0010 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc29e5a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc2415cea0 0x7fdc24157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc28e58700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc24159040 0x7fdc2415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc29e5a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc2415cea0 0x7fdc24157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34102/0 (socket says 172.21.15.38:34102) 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc29659700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc240d0010 0x7fdc240b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc28e58700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc24159040 0x7fdc2415a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:59896/0 (socket says 172.21.15.38:59896) 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.566+0000 7fdc29e5a700 1 -- 172.21.15.38:0/3897459936 learned_addr learned my addr 172.21.15.38:0/3897459936 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc29e5a700 1 -- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc240d0010 msgr2=0x7fdc240b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc29e5a700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc240d0010 0x7fdc240b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc29e5a700 1 -- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc24159040 msgr2=0x7fdc2415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc29e5a700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc24159040 0x7fdc2415a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc29e5a700 1 -- 172.21.15.38:0/3897459936 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc20009420 con 0x7fdc2415cea0 2024-08-15T17:58:47.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc29e5a700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc2415cea0 0x7fdc24157f40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c001460 tx=0x7fdc1c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:47.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc157fa700 1 -- 172.21.15.38:0/3897459936 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00a890 con 0x7fdc2415cea0 2024-08-15T17:58:47.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc157fa700 1 -- 172.21.15.38:0/3897459936 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c00a9f0 con 0x7fdc2415cea0 2024-08-15T17:58:47.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc2a65b700 1 -- 172.21.15.38:0/3897459936 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc240b1af0 con 0x7fdc2415cea0 2024-08-15T17:58:47.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc157fa700 1 -- 172.21.15.38:0/3897459936 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00ad70 con 0x7fdc2415cea0 2024-08-15T17:58:47.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc2a65b700 1 -- 172.21.15.38:0/3897459936 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc240b1dd0 con 0x7fdc2415cea0 2024-08-15T17:58:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc2a65b700 1 -- 172.21.15.38:0/3897459936 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc240643a0 con 0x7fdc2415cea0 2024-08-15T17:58:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc157fa700 1 -- 172.21.15.38:0/3897459936 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdc1c007300 con 0x7fdc2415cea0 2024-08-15T17:58:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc157fa700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc08060860 0x7fdc08062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc157fa700 1 -- 172.21.15.38:0/3897459936 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdc1c07cf50 con 0x7fdc2415cea0 2024-08-15T17:58:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.570+0000 7fdc29659700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc08060860 0x7fdc08062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.574+0000 7fdc29659700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc08060860 0x7fdc08062d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc24067ef0 tx=0x7fdc20017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:47.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.578+0000 7fdc157fa700 1 -- 172.21.15.38:0/3897459936 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdc1c04d390 con 0x7fdc2415cea0 2024-08-15T17:58:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.778+0000 7fdc2a65b700 1 -- 172.21.15.38:0/3897459936 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc2415cc50 con 0x7fdc2415cea0 2024-08-15T17:58:47.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc157fa700 1 -- 172.21.15.38:0/3897459936 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4597 (secure 0 0 0) 0x7fdc1c0509b0 con 0x7fdc2415cea0 2024-08-15T17:58:47.783 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:47.783 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2444719791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2444719791},{"type":"v1","addr":"172.21.15.38:6833","nonce":2444719791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2667173752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2667173752},{"type":"v1","addr":"172.21.15.38:6835","nonce":2667173752}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1185013072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1185013072},{"type":"v1","addr":"172.21.15.104:6835","nonce":1185013072}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:58:35.622492+0000","modified":"2024-08-15T17:58:38.885045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/3628618356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3628618356},{"type":"v1","addr":"172.21.15.104:6837","nonce":3628618356}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:58:47.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 -- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc08060860 msgr2=0x7fdc08062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc08060860 0x7fdc08062d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc24067ef0 tx=0x7fdc20017000 comp rx=0 tx=0).stop 2024-08-15T17:58:47.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 -- 172.21.15.38:0/3897459936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc2415cea0 msgr2=0x7fdc24157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc2415cea0 0x7fdc24157f40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c001460 tx=0x7fdc1c001ae0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 -- 172.21.15.38:0/3897459936 shutdown_connections 2024-08-15T17:58:47.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdc08060860 0x7fdc08062d10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc2415cea0 0x7fdc24157f40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc24159040 0x7fdc2415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 --2- 172.21.15.38:0/3897459936 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc240d0010 0x7fdc240b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 -- 172.21.15.38:0/3897459936 >> 172.21.15.38:0/3897459936 conn(0x7fdc24068250 msgr2=0x7fdc24152d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:47.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 -- 172.21.15.38:0/3897459936 shutdown_connections 2024-08-15T17:58:47.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.782+0000 7fdc2a65b700 1 -- 172.21.15.38:0/3897459936 wait complete. 2024-08-15T17:58:47.787 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-08-15T17:58:47.797 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:47.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.902+0000 7ff276f7c700 1 Processor -- start 2024-08-15T17:58:47.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.902+0000 7ff276f7c700 1 -- start start 2024-08-15T17:58:47.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.902+0000 7ff276f7c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff270154900 0x7ff2700b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.902+0000 7ff276f7c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff2700cb560 0x7ff270160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.902+0000 7ff276f7c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2700cbaa0 con 0x7ff2700b12d0 2024-08-15T17:58:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.902+0000 7ff276f7c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff270068790 con 0x7ff2700cb560 2024-08-15T17:58:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.902+0000 7ff276f7c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2701583f0 con 0x7ff270154900 2024-08-15T17:58:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff275779700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff2700b12d0 0x7ff2700cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44194/0 (socket says 172.21.15.38:44194) 2024-08-15T17:58:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff275779700 1 -- 172.21.15.38:0/2171075930 learned_addr learned my addr 172.21.15.38:0/2171075930 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff275f7a700 1 --2- 172.21.15.38:0/2171075930 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff270154900 0x7ff2700b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff27677b700 1 --2- 172.21.15.38:0/2171075930 >> v2:172.21.15.104:3300/0 conn(0x7ff2700cb560 0x7ff270160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff274f78700 1 -- 172.21.15.38:0/2171075930 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763602569 0 0) 0x7ff2700cbaa0 con 0x7ff2700b12d0 2024-08-15T17:58:47.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff274f78700 1 -- 172.21.15.38:0/2171075930 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff258003570 con 0x7ff2700b12d0 2024-08-15T17:58:47.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff274f78700 1 -- 172.21.15.38:0/2171075930 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3678053994 0 0) 0x7ff258003570 con 0x7ff2700b12d0 2024-08-15T17:58:47.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff27677b700 1 -- 172.21.15.38:0/2171075930 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff270154900 msgr2=0x7ff2700b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:47.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff27677b700 1 --2- 172.21.15.38:0/2171075930 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff270154900 0x7ff2700b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff27677b700 1 -- 172.21.15.38:0/2171075930 >> v1:172.21.15.38:6789/0 conn(0x7ff2700b12d0 legacy=0x7ff2700cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff27677b700 1 -- 172.21.15.38:0/2171075930 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2700b1700 con 0x7ff2700cb560 2024-08-15T17:58:47.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff27677b700 1 --2- 172.21.15.38:0/2171075930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2700cb560 0x7ff270160ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff2680098e0 tx=0x7ff2680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2cc264a06c4dfd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:47.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff274f78700 1 -- 172.21.15.38:0/2171075930 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2640017f0 con 0x7ff2700cb560 2024-08-15T17:58:47.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff274f78700 1 -- 172.21.15.38:0/2171075930 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268011420 con 0x7ff2700cb560 2024-08-15T17:58:47.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff274f78700 1 -- 172.21.15.38:0/2171075930 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2680117a0 con 0x7ff2700cb560 2024-08-15T17:58:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff276f7c700 1 -- 172.21.15.38:0/2171075930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2700cb560 msgr2=0x7ff270160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff276f7c700 1 --2- 172.21.15.38:0/2171075930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2700cb560 0x7ff270160ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff2680098e0 tx=0x7ff2680091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff276f7c700 1 -- 172.21.15.38:0/2171075930 shutdown_connections 2024-08-15T17:58:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff276f7c700 1 --2- 172.21.15.38:0/2171075930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2700cb560 0x7ff270160ac0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff276f7c700 1 --2- 172.21.15.38:0/2171075930 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff270154900 0x7ff2700b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff276f7c700 1 -- 172.21.15.38:0/2171075930 >> 172.21.15.38:0/2171075930 conn(0x7ff2700678d0 msgr2=0x7ff270067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff276f7c700 1 -- 172.21.15.38:0/2171075930 shutdown_connections 2024-08-15T17:58:47.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.906+0000 7ff276f7c700 1 -- 172.21.15.38:0/2171075930 wait complete. 2024-08-15T17:58:47.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.910+0000 7ff276f7c700 1 Processor -- start 2024-08-15T17:58:47.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.910+0000 7ff276f7c700 1 -- start start 2024-08-15T17:58:47.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff276f7c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2700b12d0 0x7ff27014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff276f7c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff2700cb560 0x7ff270156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff276f7c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff270154900 0x7ff2700aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff276f7c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27015aa90 con 0x7ff2700b12d0 2024-08-15T17:58:47.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff276f7c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27015d790 con 0x7ff270154900 2024-08-15T17:58:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff276f7c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27014ef20 con 0x7ff2700cb560 2024-08-15T17:58:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff275f7a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2700b12d0 0x7ff27014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff275f7a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2700b12d0 0x7ff27014e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34110/0 (socket says 172.21.15.38:34110) 2024-08-15T17:58:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff275f7a700 1 -- 172.21.15.38:0/2857689654 learned_addr learned my addr 172.21.15.38:0/2857689654 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff275779700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff2700cb560 0x7ff270156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff27677b700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff270154900 0x7ff2700aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff275f7a700 1 -- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff2700cb560 msgr2=0x7ff270156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff275f7a700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff2700cb560 0x7ff270156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff275f7a700 1 -- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff270154900 msgr2=0x7ff2700aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:47.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff275f7a700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff270154900 0x7ff2700aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff275f7a700 1 -- 172.21.15.38:0/2857689654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff268009420 con 0x7ff2700b12d0 2024-08-15T17:58:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff27677b700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff270154900 0x7ff2700aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff275f7a700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2700b12d0 0x7ff27014e9e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ec80 tx=0x7ff2600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff26dffb700 1 -- 172.21.15.38:0/2857689654 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26000c9d0 con 0x7ff2700b12d0 2024-08-15T17:58:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff26dffb700 1 -- 172.21.15.38:0/2857689654 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26000cb30 con 0x7ff2700b12d0 2024-08-15T17:58:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff276f7c700 1 -- 172.21.15.38:0/2857689654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27015cb80 con 0x7ff2700b12d0 2024-08-15T17:58:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff26dffb700 1 -- 172.21.15.38:0/2857689654 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260013470 con 0x7ff2700b12d0 2024-08-15T17:58:47.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff276f7c700 1 -- 172.21.15.38:0/2857689654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27015d140 con 0x7ff2700b12d0 2024-08-15T17:58:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.914+0000 7ff276f7c700 1 -- 172.21.15.38:0/2857689654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27006b0c0 con 0x7ff2700b12d0 2024-08-15T17:58:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.918+0000 7ff26dffb700 1 -- 172.21.15.38:0/2857689654 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff2600137e0 con 0x7ff2700b12d0 2024-08-15T17:58:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.918+0000 7ff26dffb700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff25c060860 0x7ff25c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.918+0000 7ff26dffb700 1 -- 172.21.15.38:0/2857689654 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff26007cf30 con 0x7ff2700b12d0 2024-08-15T17:58:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.918+0000 7ff275779700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff25c060860 0x7ff25c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:47.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.918+0000 7ff275779700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff25c060860 0x7ff25c062d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff264007fd0 tx=0x7ff264007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:47.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:47.926+0000 7ff26dffb700 1 -- 172.21.15.38:0/2857689654 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff26004d380 con 0x7ff2700b12d0 2024-08-15T17:58:48.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.094+0000 7ff276f7c700 1 -- 172.21.15.38:0/2857689654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2700643a0 con 0x7ff2700b12d0 2024-08-15T17:58:48.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.094+0000 7ff26dffb700 1 -- 172.21.15.38:0/2857689654 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7ff260017090 con 0x7ff2700b12d0 2024-08-15T17:58:48.098 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:48.098 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 -- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff25c060860 msgr2=0x7ff25c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff25c060860 0x7ff25c062d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff264007fd0 tx=0x7ff264007e90 comp rx=0 tx=0).stop 2024-08-15T17:58:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 -- 172.21.15.38:0/2857689654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2700b12d0 msgr2=0x7ff27014e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2700b12d0 0x7ff27014e9e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ec80 tx=0x7ff2600091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 -- 172.21.15.38:0/2857689654 shutdown_connections 2024-08-15T17:58:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff25c060860 0x7ff25c062d10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff270154900 0x7ff2700aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff2700cb560 0x7ff270156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 --2- 172.21.15.38:0/2857689654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2700b12d0 0x7ff27014e9e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 -- 172.21.15.38:0/2857689654 >> 172.21.15.38:0/2857689654 conn(0x7ff2700678d0 msgr2=0x7ff270153730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 -- 172.21.15.38:0/2857689654 shutdown_connections 2024-08-15T17:58:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.098+0000 7ff276f7c700 1 -- 172.21.15.38:0/2857689654 wait complete. 2024-08-15T17:58:48.113 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:58:48.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.218+0000 7f9ec595f700 1 Processor -- start 2024-08-15T17:58:48.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.218+0000 7f9ec595f700 1 -- start start 2024-08-15T17:58:48.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9ec0159ce0 0x7f9ec015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ec0153430 0x7f9ec0153880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec01578c0 con 0x7f9ec00b0960 2024-08-15T17:58:48.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec01575c0 con 0x7f9ec0159ce0 2024-08-15T17:58:48.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec00b16c0 con 0x7f9ec0153430 2024-08-15T17:58:48.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec495d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9ec0159ce0 0x7f9ec015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec495d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9ec0159ce0 0x7f9ec015a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34118/0 (socket says 172.21.15.38:34118) 2024-08-15T17:58:48.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec495d700 1 -- 172.21.15.38:0/1385126941 learned_addr learned my addr 172.21.15.38:0/1385126941 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:48.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ebffff700 1 --2- 172.21.15.38:0/1385126941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ec0153430 0x7f9ec0153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ebf7fe700 1 -- 172.21.15.38:0/1385126941 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477115656 0 0) 0x7f9ec01578c0 con 0x7f9ec00b0960 2024-08-15T17:58:48.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ebf7fe700 1 -- 172.21.15.38:0/1385126941 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb0003570 con 0x7f9ec00b0960 2024-08-15T17:58:48.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec495d700 1 -- 172.21.15.38:0/1385126941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ec0153430 msgr2=0x7f9ec0153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:48.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec495d700 1 --2- 172.21.15.38:0/1385126941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ec0153430 0x7f9ec0153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec495d700 1 -- 172.21.15.38:0/1385126941 >> v1:172.21.15.104:6789/0 conn(0x7f9ec00b0960 legacy=0x7f9ec00b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec495d700 1 -- 172.21.15.38:0/1385126941 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec00b19a0 con 0x7f9ec0159ce0 2024-08-15T17:58:48.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ebffff700 1 --2- 172.21.15.38:0/1385126941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ec0153430 0x7f9ec0153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:48.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec495d700 1 --2- 172.21.15.38:0/1385126941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ec0159ce0 0x7f9ec015a0b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40099b0 tx=0x7f9eb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838855b44946347e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:48.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ebf7fe700 1 -- 172.21.15.38:0/1385126941 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400b680 con 0x7f9ec0159ce0 2024-08-15T17:58:48.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ebf7fe700 1 -- 172.21.15.38:0/1385126941 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb400b7e0 con 0x7f9ec0159ce0 2024-08-15T17:58:48.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ebf7fe700 1 -- 172.21.15.38:0/1385126941 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400bb80 con 0x7f9ec0159ce0 2024-08-15T17:58:48.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 -- 172.21.15.38:0/1385126941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ec0159ce0 msgr2=0x7f9ec015a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 --2- 172.21.15.38:0/1385126941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ec0159ce0 0x7f9ec015a0b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40099b0 tx=0x7f9eb4009250 comp rx=0 tx=0).stop 2024-08-15T17:58:48.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 -- 172.21.15.38:0/1385126941 shutdown_connections 2024-08-15T17:58:48.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 --2- 172.21.15.38:0/1385126941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ec0153430 0x7f9ec0153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 --2- 172.21.15.38:0/1385126941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ec0159ce0 0x7f9ec015a0b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 -- 172.21.15.38:0/1385126941 >> 172.21.15.38:0/1385126941 conn(0x7f9ec00c9840 msgr2=0x7f9ec00c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:48.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 -- 172.21.15.38:0/1385126941 shutdown_connections 2024-08-15T17:58:48.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.222+0000 7f9ec595f700 1 -- 172.21.15.38:0/1385126941 wait complete. 2024-08-15T17:58:48.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec595f700 1 Processor -- start 2024-08-15T17:58:48.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec595f700 1 -- start start 2024-08-15T17:58:48.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec595f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ec00b0960 0x7f9ec006b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec595f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ec0153430 0x7f9ec013d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec595f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ec0159ce0 0x7f9ec0068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec595f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec0153bb0 con 0x7f9ec0159ce0 2024-08-15T17:58:48.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec595f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec0155c20 con 0x7f9ec00b0960 2024-08-15T17:58:48.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec595f700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec006bbe0 con 0x7f9ec0153430 2024-08-15T17:58:48.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec515e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ec0159ce0 0x7f9ec0068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec515e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ec0159ce0 0x7f9ec0068cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34134/0 (socket says 172.21.15.38:34134) 2024-08-15T17:58:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec515e700 1 -- 172.21.15.38:0/443576331 learned_addr learned my addr 172.21.15.38:0/443576331 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ec495d700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ec00b0960 0x7f9ec006b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.226+0000 7f9ebffff700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ec0153430 0x7f9ec013d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ec515e700 1 -- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ec0153430 msgr2=0x7f9ec013d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ec515e700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ec0153430 0x7f9ec013d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ec515e700 1 -- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ec00b0960 msgr2=0x7f9ec006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ec515e700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ec00b0960 0x7f9ec006b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ec515e700 1 -- 172.21.15.38:0/443576331 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac009420 con 0x7f9ec0159ce0 2024-08-15T17:58:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ec515e700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ec0159ce0 0x7f9ec0068cf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c910 tx=0x7f9eb80027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ebcff9700 1 -- 172.21.15.38:0/443576331 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8012630 con 0x7f9ec0159ce0 2024-08-15T17:58:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ebcff9700 1 -- 172.21.15.38:0/443576331 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb80127b0 con 0x7f9ec0159ce0 2024-08-15T17:58:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ec595f700 1 -- 172.21.15.38:0/443576331 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb40094f0 con 0x7f9ec0159ce0 2024-08-15T17:58:48.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ebcff9700 1 -- 172.21.15.38:0/443576331 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8012b70 con 0x7f9ec0159ce0 2024-08-15T17:58:48.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ec595f700 1 -- 172.21.15.38:0/443576331 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec01552f0 con 0x7f9ec0159ce0 2024-08-15T17:58:48.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ec595f700 1 -- 172.21.15.38:0/443576331 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec0155d90 con 0x7f9ec0159ce0 2024-08-15T17:58:48.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ebcff9700 1 -- 172.21.15.38:0/443576331 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9eb801b400 con 0x7f9ec0159ce0 2024-08-15T17:58:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ebcff9700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9ea4060890 0x7f9ea4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ebcff9700 1 -- 172.21.15.38:0/443576331 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9eb807cec0 con 0x7f9ec0159ce0 2024-08-15T17:58:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.230+0000 7f9ec495d700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9ea4060890 0x7f9ea4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.234+0000 7f9ec495d700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9ea4060890 0x7f9ea4062d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40099b0 tx=0x7f9eb4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:48.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.238+0000 7f9ebcff9700 1 -- 172.21.15.38:0/443576331 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9eb804d3c0 con 0x7f9ec0159ce0 2024-08-15T17:58:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.438+0000 7f9ec595f700 1 -- 172.21.15.38:0/443576331 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ec00ac770 con 0x7f9ec0159ce0 2024-08-15T17:58:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ebcff9700 1 -- 172.21.15.38:0/443576331 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f9eb8016090 con 0x7f9ec0159ce0 2024-08-15T17:58:48.443 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:48.443 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2444719791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2444719791},{"type":"v1","addr":"172.21.15.38:6833","nonce":2444719791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2667173752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2667173752},{"type":"v1","addr":"172.21.15.38:6835","nonce":2667173752}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1185013072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1185013072},{"type":"v1","addr":"172.21.15.104:6835","nonce":1185013072}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:58:35.622492+0000","modified":"2024-08-15T17:58:38.885045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/3628618356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3628618356},{"type":"v1","addr":"172.21.15.104:6837","nonce":3628618356}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:58:48.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 -- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9ea4060890 msgr2=0x7f9ea4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9ea4060890 0x7f9ea4062d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40099b0 tx=0x7f9eb4017000 comp rx=0 tx=0).stop 2024-08-15T17:58:48.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 -- 172.21.15.38:0/443576331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ec0159ce0 msgr2=0x7f9ec0068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ec0159ce0 0x7f9ec0068cf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c910 tx=0x7f9eb80027a0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 -- 172.21.15.38:0/443576331 shutdown_connections 2024-08-15T17:58:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9ea4060890 0x7f9ea4062d40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ec0159ce0 0x7f9ec0068cf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ec0153430 0x7f9ec013d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 --2- 172.21.15.38:0/443576331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ec00b0960 0x7f9ec006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 -- 172.21.15.38:0/443576331 >> 172.21.15.38:0/443576331 conn(0x7f9ec00c9840 msgr2=0x7f9ec0152640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 -- 172.21.15.38:0/443576331 shutdown_connections 2024-08-15T17:58:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.442+0000 7f9ec595f700 1 -- 172.21.15.38:0/443576331 wait complete. 2024-08-15T17:58:48.447 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-08-15T17:58:48.860 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-15T17:58:35.622492+0000', 'modified': '2024-08-15T17:58:38.885045+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6837/3628618356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 3628618356}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 3628618356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-08-15T17:58:48.860 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-15T17:58:48.860 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-08-15T17:58:48.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.978+0000 7f1ab7fff700 1 Processor -- start 2024-08-15T17:58:48.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.978+0000 7f1ab7fff700 1 -- start start 2024-08-15T17:58:48.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1ab806e620 0x7f1ab80b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1ab80b1630 0x7f1ab8141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab8141a00 con 0x7f1ab80b0d20 2024-08-15T17:58:48.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab8140b10 con 0x7f1ab806e620 2024-08-15T17:58:48.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab8140df0 con 0x7f1ab80b1630 2024-08-15T17:58:48.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab67fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1ab806e620 0x7f1ab80b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab67fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1ab806e620 0x7f1ab80b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34140/0 (socket says 172.21.15.38:34140) 2024-08-15T17:58:48.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab67fc700 1 -- 172.21.15.38:0/2684977993 learned_addr learned my addr 172.21.15.38:0/2684977993 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:48.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab77fe700 1 --2- 172.21.15.38:0/2684977993 >> v2:172.21.15.104:3300/0 conn(0x7f1ab80b1630 0x7f1ab8141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab67fc700 1 -- 172.21.15.38:0/2684977993 >> v2:172.21.15.104:3300/0 conn(0x7f1ab80b1630 msgr2=0x7f1ab8141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/2684977993 >> v2:172.21.15.104:3300/0 conn(0x7f1ab80b1630 0x7f1ab8141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab67fc700 1 -- 172.21.15.38:0/2684977993 >> v1:172.21.15.104:6789/0 conn(0x7f1ab80b0d20 legacy=0x7f1ab806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab67fc700 1 -- 172.21.15.38:0/2684977993 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab81410d0 con 0x7f1ab806e620 2024-08-15T17:58:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/2684977993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ab806e620 0x7f1ab80b10f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ab806eb70 tx=0x7f1aac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78157e4eb5dab342 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab5ffb700 1 -- 172.21.15.38:0/2684977993 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac00b860 con 0x7f1ab806e620 2024-08-15T17:58:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab5ffb700 1 -- 172.21.15.38:0/2684977993 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac00b9c0 con 0x7f1ab806e620 2024-08-15T17:58:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab5ffb700 1 -- 172.21.15.38:0/2684977993 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac00bd60 con 0x7f1ab806e620 2024-08-15T17:58:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 -- 172.21.15.38:0/2684977993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ab806e620 msgr2=0x7f1ab80b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 --2- 172.21.15.38:0/2684977993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ab806e620 0x7f1ab80b10f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ab806eb70 tx=0x7f1aac009250 comp rx=0 tx=0).stop 2024-08-15T17:58:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 -- 172.21.15.38:0/2684977993 shutdown_connections 2024-08-15T17:58:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 --2- 172.21.15.38:0/2684977993 >> v2:172.21.15.104:3300/0 conn(0x7f1ab80b1630 0x7f1ab8141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 --2- 172.21.15.38:0/2684977993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ab806e620 0x7f1ab80b10f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 -- 172.21.15.38:0/2684977993 >> 172.21.15.38:0/2684977993 conn(0x7f1ab80678d0 msgr2=0x7f1ab8067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 -- 172.21.15.38:0/2684977993 shutdown_connections 2024-08-15T17:58:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.982+0000 7f1ab7fff700 1 -- 172.21.15.38:0/2684977993 wait complete. 2024-08-15T17:58:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab7fff700 1 Processor -- start 2024-08-15T17:58:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab7fff700 1 -- start start 2024-08-15T17:58:48.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab7fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ab806e620 0x7f1ab8170770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ab80b0d20 0x7f1ab8145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab80b1630 0x7f1ab815fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab7fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80a7280 con 0x7f1ab80b0d20 2024-08-15T17:58:48.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab7fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80a6200 con 0x7f1ab80b1630 2024-08-15T17:58:48.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab7fff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab81465a0 con 0x7f1ab806e620 2024-08-15T17:58:48.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ab80b0d20 0x7f1ab8145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ab80b0d20 0x7f1ab8145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34152/0 (socket says 172.21.15.38:34152) 2024-08-15T17:58:48.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab67fc700 1 -- 172.21.15.38:0/4147403923 learned_addr learned my addr 172.21.15.38:0/4147403923 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:48.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ab806e620 0x7f1ab8170770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab77fe700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab80b1630 0x7f1ab815fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab67fc700 1 -- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ab806e620 msgr2=0x7f1ab8170770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ab806e620 0x7f1ab8170770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab67fc700 1 -- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab80b1630 msgr2=0x7f1ab815fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:48.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab80b1630 0x7f1ab815fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab67fc700 1 -- 172.21.15.38:0/4147403923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac0094f0 con 0x7f1ab80b0d20 2024-08-15T17:58:48.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.986+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ab80b0d20 0x7f1ab8145460 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1aac006fd0 tx=0x7f1aac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:48.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.990+0000 7f1a9effd700 1 -- 172.21.15.38:0/4147403923 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac024440 con 0x7f1ab80b0d20 2024-08-15T17:58:48.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.990+0000 7f1a9effd700 1 -- 172.21.15.38:0/4147403923 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac0245a0 con 0x7f1ab80b0d20 2024-08-15T17:58:48.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.990+0000 7f1ab7fff700 1 -- 172.21.15.38:0/4147403923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab80a55f0 con 0x7f1ab80b0d20 2024-08-15T17:58:48.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.990+0000 7f1a9effd700 1 -- 172.21.15.38:0/4147403923 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac024960 con 0x7f1ab80b0d20 2024-08-15T17:58:48.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.990+0000 7f1ab7fff700 1 -- 172.21.15.38:0/4147403923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab80a5bb0 con 0x7f1ab80b0d20 2024-08-15T17:58:48.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.990+0000 7f1a9effd700 1 -- 172.21.15.38:0/4147403923 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1aac01a420 con 0x7f1ab80b0d20 2024-08-15T17:58:48.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.990+0000 7f1a9effd700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1aa4060890 0x7f1aa4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1a9effd700 1 -- 172.21.15.38:0/4147403923 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1aac07e890 con 0x7f1ab80b0d20 2024-08-15T17:58:48.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1aa4060890 0x7f1aa4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1aa4060890 0x7f1aa4062d40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80039e0 tx=0x7f1aa8007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:48.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1abf777740 1 Processor -- start 2024-08-15T17:58:48.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1abf777740 1 -- start start 2024-08-15T17:58:48.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1abf777740 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x11b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1abf777740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x11b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1abf777740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xfc9790 0x11b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1abf777740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x112aba0 con 0xf7e710 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1abf777740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1126b40 con 0xfc9790 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1abf777740 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x10ec0a0 con 0xf7eae0 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x11b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x11b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34154/0 (socket says 172.21.15.38:34154) 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab67fc700 1 -- 172.21.15.38:0/2725709672 learned_addr learned my addr 172.21.15.38:0/2725709672 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab77fe700 1 --2- 172.21.15.38:0/2725709672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xfc9790 0x11b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/2725709672 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x11b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab67fc700 1 -- 172.21.15.38:0/2725709672 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 msgr2=0x11b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/2725709672 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x11b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab67fc700 1 -- 172.21.15.38:0/2725709672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xfc9790 msgr2=0x11b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/2725709672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xfc9790 0x11b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:48.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab67fc700 1 -- 172.21.15.38:0/2725709672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xfca9e0 con 0xf7e710 2024-08-15T17:58:49.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/2725709672 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x11b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:49.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/2725709672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x11b2710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1aac07cd10 tx=0x7f1aac07cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c01bd4ffc7cfb3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:49.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.994+0000 7f1a97fff700 1 -- 172.21.15.38:0/2725709672 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac053040 con 0xf7e710 2024-08-15T17:58:49.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1a97fff700 1 -- 172.21.15.38:0/2725709672 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac055b90 con 0xf7e710 2024-08-15T17:58:49.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1a97fff700 1 -- 172.21.15.38:0/2725709672 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0598e0 con 0xf7e710 2024-08-15T17:58:49.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 -- 172.21.15.38:0/2725709672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 msgr2=0x11b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:49.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 --2- 172.21.15.38:0/2725709672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x11b2710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1aac07cd10 tx=0x7f1aac07cc70 comp rx=0 tx=0).stop 2024-08-15T17:58:49.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 -- 172.21.15.38:0/2725709672 shutdown_connections 2024-08-15T17:58:49.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 --2- 172.21.15.38:0/2725709672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xfc9790 0x11b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 --2- 172.21.15.38:0/2725709672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x11b2710 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 --2- 172.21.15.38:0/2725709672 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x11b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 -- 172.21.15.38:0/2725709672 >> 172.21.15.38:0/2725709672 conn(0xf7eeb0 msgr2=0x11b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:49.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 -- 172.21.15.38:0/2725709672 shutdown_connections 2024-08-15T17:58:49.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 -- 172.21.15.38:0/2725709672 wait complete. 2024-08-15T17:58:49.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 -- start start 2024-08-15T17:58:49.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 Processor -- start 2024-08-15T17:58:49.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 0x11ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x11fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xfc9790 0x11b9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x10ebd40 con 0xfc9790 2024-08-15T17:58:49.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x10c4970 con 0xf7e710 2024-08-15T17:58:49.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x112b480 con 0xf7eae0 2024-08-15T17:58:49.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1abf777740 10 client.0 _ll_register_callbacks cb 0x10c8f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T17:58:49.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1ab77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xfc9790 0x11b9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1ab77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xfc9790 0x11b9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34156/0 (socket says 172.21.15.38:34156) 2024-08-15T17:58:49.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1ab77fe700 1 -- 172.21.15.38:0/3028956313 learned_addr learned my addr 172.21.15.38:0/3028956313 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x11fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:48.998+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 0x11ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1ab77fe700 1 -- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 msgr2=0x11fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1ab77fe700 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x11fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1ab77fe700 1 -- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 msgr2=0x11ffaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1ab77fe700 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 0x11ffaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1ab77fe700 1 -- 172.21.15.38:0/3028956313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab80a5bb0 con 0xfc9790 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 0x11ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1ab77fe700 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xfc9790 0x11b9fd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0007e90 tx=0x7f1ab00097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbee8865a2c31192 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1a957fa700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1a957fa700 1 -- 172.21.15.38:0/3028956313 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0002de0 con 0xfc9790 2024-08-15T17:58:49.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1a957fa700 1 -- 172.21.15.38:0/3028956313 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ab0017470 con 0xfc9790 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1abf777740 1 -- 172.21.15.38:0/3028956313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xf8c260 con 0xfc9790 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1a957fa700 1 -- 172.21.15.38:0/3028956313 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ab0017840 con 0xfc9790 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1abf777740 10 client.4671 fetch_fsmap learned FSMap version 7 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1abf777740 1 -- 172.21.15.38:0/3028956313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x10f4370 con 0xfc9790 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1a957fa700 1 -- 172.21.15.38:0/3028956313 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xf8c260 con 0xfc9790 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1a957fa700 1 -- 172.21.15.38:0/3028956313 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1ab00179a0 con 0xfc9790 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1abf777740 10 client.4671 fetch_fsmap finished waiting for FSMap version 7 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1abf777740 10 client.4671 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1abf777740 20 client.4671 populate_metadata read hostname 'smithi038' 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1abf777740 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:6836/3628618356,v1:172.21.15.104:6837/3628618356] conn(0x1208c30 0x120b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1abf777740 4 client.4671 mds_command: new command op to 4294 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1abf777740 1 -- 172.21.15.38:0/3028956313 --> [v2:172.21.15.104:6836/3628618356,v1:172.21.15.104:6837/3628618356] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xfd64e0 con 0x1208c30 2024-08-15T17:58:49.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:6836/3628618356,v1:172.21.15.104:6837/3628618356] conn(0x1208c30 0x120b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:6836/3628618356,v1:172.21.15.104:6837/3628618356] conn(0x1208c30 0x120b010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b179f1a0b7a37d server_cookie=8914ef1156652611 in_seq=0 out_seq=0 2024-08-15T17:58:49.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.002+0000 7f1a957fa700 10 client.4671 ms_handle_connect on v2:172.21.15.104:6836/3628618356 2024-08-15T17:58:49.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1a957fa700 1 -- 172.21.15.38:0/3028956313 <== mds.0 v2:172.21.15.104:6836/3628618356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0xfd64e0 con 0x1208c30 2024-08-15T17:58:49.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1a957fa700 10 client.4671 handle_command_reply: tid=0 2024-08-15T17:58:49.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 1 client.4671 shutdown 2024-08-15T17:58:49.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 1 -- 172.21.15.38:0/3028956313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xfc9790 msgr2=0x11b9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:49.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xfc9790 0x11b9fd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0007e90 tx=0x7f1ab00097b0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 1 -- 172.21.15.38:0/3028956313 shutdown_connections 2024-08-15T17:58:49.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:6836/3628618356,v1:172.21.15.104:6837/3628618356] conn(0x1208c30 0x120b010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xfc9790 0x11b9fd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x11fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 1 --2- 172.21.15.38:0/3028956313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 0x11ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1a957fa700 0 client.4671 ms_handle_reset on v2:172.21.15.104:6836/3628618356 2024-08-15T17:58:49.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 1 -- 172.21.15.38:0/3028956313 >> 172.21.15.38:0/3028956313 conn(0xf7eeb0 msgr2=0x11b7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:49.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 1 -- 172.21.15.38:0/3028956313 shutdown_connections 2024-08-15T17:58:49.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 1 -- 172.21.15.38:0/3028956313 wait complete. 2024-08-15T17:58:49.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.006+0000 7f1abf777740 20 client.4671 trim_cache size 0 max 16384 2024-08-15T17:58:49.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1abf777740 1 Processor -- start 2024-08-15T17:58:49.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1abf777740 1 -- start start 2024-08-15T17:58:49.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1abf777740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 0x1253b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1abf777740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7eae0 0x1254070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1abf777740 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xfc9790 0x12545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1abf777740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x10f09c0 con 0xf7eae0 2024-08-15T17:58:49.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1abf777740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0xffb650 con 0xf7e710 2024-08-15T17:58:49.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1abf777740 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x10d5140 con 0xfc9790 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7eae0 0x1254070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7eae0 0x1254070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34162/0 (socket says 172.21.15.38:34162) 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab67fc700 1 -- 172.21.15.38:0/1094803604 learned_addr learned my addr 172.21.15.38:0/1094803604 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/1094803604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 0x1253b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab77fe700 1 --2- 172.21.15.38:0/1094803604 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xfc9790 0x12545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab67fc700 1 -- 172.21.15.38:0/1094803604 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xfc9790 msgr2=0x12545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/1094803604 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xfc9790 0x12545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab67fc700 1 -- 172.21.15.38:0/1094803604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 msgr2=0x1253b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/1094803604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 0x1253b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab67fc700 1 -- 172.21.15.38:0/1094803604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10dad80 con 0xf7eae0 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/1094803604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 0x1253b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/1094803604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7eae0 0x1254070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1aac014040 tx=0x7f1aac066000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4e84c346a3e1bb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:49.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1a9cff9700 1 -- 172.21.15.38:0/1094803604 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0598e0 con 0xf7eae0 2024-08-15T17:58:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1a9cff9700 1 -- 172.21.15.38:0/1094803604 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac052900 con 0xf7eae0 2024-08-15T17:58:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.026+0000 7f1a9cff9700 1 -- 172.21.15.38:0/1094803604 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac024ac0 con 0xf7eae0 2024-08-15T17:58:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 -- 172.21.15.38:0/1094803604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7eae0 msgr2=0x1254070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 --2- 172.21.15.38:0/1094803604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7eae0 0x1254070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1aac014040 tx=0x7f1aac066000 comp rx=0 tx=0).stop 2024-08-15T17:58:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 -- 172.21.15.38:0/1094803604 shutdown_connections 2024-08-15T17:58:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 --2- 172.21.15.38:0/1094803604 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xfc9790 0x12545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 --2- 172.21.15.38:0/1094803604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7eae0 0x1254070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 --2- 172.21.15.38:0/1094803604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xf7e710 0x1253b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 -- 172.21.15.38:0/1094803604 >> 172.21.15.38:0/1094803604 conn(0xf7eeb0 msgr2=0x11b8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 -- 172.21.15.38:0/1094803604 shutdown_connections 2024-08-15T17:58:49.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 -- 172.21.15.38:0/1094803604 wait complete. 2024-08-15T17:58:49.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 -- start start 2024-08-15T17:58:49.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 Processor -- start 2024-08-15T17:58:49.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x12378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x1209000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xfc9790 0x1209540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x1124500 con 0xf7e710 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.030+0000 7f1abf777740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0xff77c0 con 0xfc9790 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x12378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1abf777740 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x115cbb0 con 0xf7eae0 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x12378e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34166/0 (socket says 172.21.15.38:34166) 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x1209000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xfc9790 0x1209540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab6ffd700 1 -- 172.21.15.38:0/397495466 learned_addr learned my addr 172.21.15.38:0/397495466 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1abf777740 10 client.0 _ll_register_callbacks cb 0xf4a9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab6ffd700 1 -- 172.21.15.38:0/397495466 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 msgr2=0x1209000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/397495466 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x1209000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab6ffd700 1 -- 172.21.15.38:0/397495466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xfc9790 msgr2=0x1209540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/397495466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xfc9790 0x1209540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab6ffd700 1 -- 172.21.15.38:0/397495466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xfca9e0 con 0xf7e710 2024-08-15T17:58:49.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab6ffd700 1 --2- 172.21.15.38:0/397495466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x12378e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8003f10 tx=0x7f1aa8029040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ccc5cc7a68ef212 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:49.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1a957fa700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-08-15T17:58:49.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1a957fa700 1 -- 172.21.15.38:0/397495466 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8027c00 con 0xf7e710 2024-08-15T17:58:49.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1a957fa700 1 -- 172.21.15.38:0/397495466 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1aa801c040 con 0xf7e710 2024-08-15T17:58:49.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1abf777740 1 -- 172.21.15.38:0/397495466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1aac055b90 con 0xf7e710 2024-08-15T17:58:49.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1a957fa700 1 -- 172.21.15.38:0/397495466 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aa8023460 con 0xf7e710 2024-08-15T17:58:49.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1abf777740 10 client.4677 fetch_fsmap learned FSMap version 7 2024-08-15T17:58:49.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1abf777740 1 -- 172.21.15.38:0/397495466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x10cf130 con 0xf7e710 2024-08-15T17:58:49.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1a957fa700 1 -- 172.21.15.38:0/397495466 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac055b90 con 0xf7e710 2024-08-15T17:58:49.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1a957fa700 1 -- 172.21.15.38:0/397495466 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1aa80235c0 con 0xf7e710 2024-08-15T17:58:49.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1abf777740 10 client.4677 fetch_fsmap finished waiting for FSMap version 7 2024-08-15T17:58:49.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1abf777740 10 client.4677 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-15T17:58:49.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1abf777740 20 client.4677 populate_metadata read hostname 'smithi038' 2024-08-15T17:58:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1abf777740 1 --2- 172.21.15.38:0/397495466 >> [v2:172.21.15.104:6836/3628618356,v1:172.21.15.104:6837/3628618356] conn(0x1209a80 0x1257540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1abf777740 4 client.4677 mds_command: new command op to 4294 tid=0[{"prefix": "status"}] 2024-08-15T17:58:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1abf777740 1 -- 172.21.15.38:0/397495466 --> [v2:172.21.15.104:6836/3628618356,v1:172.21.15.104:6837/3628618356] -- command(tid 0: {"prefix": "status"}) v1 -- 0xffb260 con 0x1209a80 2024-08-15T17:58:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/397495466 >> [v2:172.21.15.104:6836/3628618356,v1:172.21.15.104:6837/3628618356] conn(0x1209a80 0x1257540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1a957fa700 10 client.4677 ms_handle_connect on v2:172.21.15.104:6836/3628618356 2024-08-15T17:58:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.034+0000 7f1ab67fc700 1 --2- 172.21.15.38:0/397495466 >> [v2:172.21.15.104:6836/3628618356,v1:172.21.15.104:6837/3628618356] conn(0x1209a80 0x1257540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e7285ab6781c6f5 server_cookie=29c56bac192c5bd in_seq=0 out_seq=0 2024-08-15T17:58:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1a957fa700 1 -- 172.21.15.38:0/397495466 <== mds.0 v2:172.21.15.104:6836/3628618356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0xffb260 con 0x1209a80 2024-08-15T17:58:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1a957fa700 10 client.4677 handle_command_reply: tid=0 2024-08-15T17:58:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 1 client.4677 shutdown 2024-08-15T17:58:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 1 -- 172.21.15.38:0/397495466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 msgr2=0x12378e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 1 --2- 172.21.15.38:0/397495466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x12378e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8003f10 tx=0x7f1aa8029040 comp rx=0 tx=0).stop 2024-08-15T17:58:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 1 -- 172.21.15.38:0/397495466 shutdown_connections 2024-08-15T17:58:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 1 --2- 172.21.15.38:0/397495466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xfc9790 0x1209540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 1 --2- 172.21.15.38:0/397495466 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0xf7eae0 0x1209000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 1 --2- 172.21.15.38:0/397495466 >> [v2:172.21.15.104:6836/3628618356,v1:172.21.15.104:6837/3628618356] conn(0x1209a80 0x1257540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 1 --2- 172.21.15.38:0/397495466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0xf7e710 0x12378e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1a957fa700 0 client.4677 ms_handle_reset on v2:172.21.15.104:6836/3628618356 2024-08-15T17:58:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 1 -- 172.21.15.38:0/397495466 >> 172.21.15.38:0/397495466 conn(0xf7eeb0 msgr2=0x11fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 1 -- 172.21.15.38:0/397495466 shutdown_connections 2024-08-15T17:58:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 1 -- 172.21.15.38:0/397495466 wait complete. 2024-08-15T17:58:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1abf777740 20 client.4677 trim_cache size 0 max 16384 2024-08-15T17:58:49.043 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T17:58:49.043 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "3c800201-a1b3-4c6d-887f-37b16c41eb0e", 2024-08-15T17:58:49.043 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-08-15T17:58:49.043 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4294, 2024-08-15T17:58:49.043 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-08-15T17:58:49.043 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-08-15T17:58:49.043 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-08-15T17:58:49.044 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 12.763401943, 2024-08-15T17:58:49.044 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 7, 2024-08-15T17:58:49.044 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 21, 2024-08-15T17:58:49.044 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 21, 2024-08-15T17:58:49.044 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 19.491726366999998 2024-08-15T17:58:49.044 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T17:58:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 -- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1aa4060890 msgr2=0x7f1aa4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1aa4060890 0x7f1aa4062d40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80039e0 tx=0x7f1aa8007ec0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 -- 172.21.15.38:0/4147403923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ab80b0d20 msgr2=0x7f1ab8145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ab80b0d20 0x7f1ab8145460 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1aac006fd0 tx=0x7f1aac017000 comp rx=0 tx=0).stop 2024-08-15T17:58:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 -- 172.21.15.38:0/4147403923 shutdown_connections 2024-08-15T17:58:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1aa4060890 0x7f1aa4062d40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab80b1630 0x7f1ab815fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ab80b0d20 0x7f1ab8145460 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 --2- 172.21.15.38:0/4147403923 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ab806e620 0x7f1ab8170770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 -- 172.21.15.38:0/4147403923 >> 172.21.15.38:0/4147403923 conn(0x7f1ab80678d0 msgr2=0x7f1ab815db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 -- 172.21.15.38:0/4147403923 shutdown_connections 2024-08-15T17:58:49.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.038+0000 7f1ab7fff700 1 -- 172.21.15.38:0/4147403923 wait complete. 2024-08-15T17:58:49.060 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-08-15T17:58:49.084 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-08-15T17:58:49.084 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-08-15T17:58:49.084 INFO:tasks.ceph_fuse:client.0 config is {} 2024-08-15T17:58:49.085 INFO:tasks.ceph_fuse:client.2 config is {} 2024-08-15T17:58:49.085 INFO:tasks.ceph_fuse:client.1 config is {} 2024-08-15T17:58:49.085 INFO:tasks.ceph_fuse:client.3 config is {} 2024-08-15T17:58:49.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.085 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-08-15T17:58:49.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.094 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link delete ceph-brx 2024-08-15T17:58:49.114 INFO:teuthology.orchestra.run.smithi104.stderr:Cannot find device "ceph-brx" 2024-08-15T17:58:49.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:58:49.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.115 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-08-15T17:58:49.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.123 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-08-15T17:58:49.142 INFO:teuthology.orchestra.run.smithi038.stderr:Cannot find device "ceph-brx" 2024-08-15T17:58:49.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:58:49.144 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-08-15T17:58:49.145 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:58:49.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7359e700 1 Processor -- start 2024-08-15T17:58:49.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7359e700 1 -- start start 2024-08-15T17:58:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7359e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2f741553e0 0x7f2f740af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7359e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f741557b0 0x7f2f740b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7359e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2f740686a0 0x7f2f74068b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7359e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f74069060 con 0x7f2f741553e0 2024-08-15T17:58:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7359e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f74155c00 con 0x7f2f740686a0 2024-08-15T17:58:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7359e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f74155e30 con 0x7f2f741557b0 2024-08-15T17:58:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7259c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2f741553e0 0x7f2f740af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7259c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2f741553e0 0x7f2f740af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34172/0 (socket says 172.21.15.38:34172) 2024-08-15T17:58:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7259c700 1 -- 172.21.15.38:0/2391628570 learned_addr learned my addr 172.21.15.38:0/2391628570 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f6bfff700 1 --2- 172.21.15.38:0/2391628570 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f741557b0 0x7f2f740b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f72d9d700 1 --2- 172.21.15.38:0/2391628570 >> v2:172.21.15.104:3300/0 conn(0x7f2f740686a0 0x7f2f74068b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7259c700 1 -- 172.21.15.38:0/2391628570 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f741557b0 msgr2=0x7f2f740b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7259c700 1 --2- 172.21.15.38:0/2391628570 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f741557b0 0x7f2f740b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7259c700 1 -- 172.21.15.38:0/2391628570 >> v2:172.21.15.104:3300/0 conn(0x7f2f740686a0 msgr2=0x7f2f74068b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7259c700 1 --2- 172.21.15.38:0/2391628570 >> v2:172.21.15.104:3300/0 conn(0x7f2f740686a0 0x7f2f74068b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.262+0000 7f2f7259c700 1 -- 172.21.15.38:0/2391628570 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f74156110 con 0x7f2f741553e0 2024-08-15T17:58:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f72d9d700 1 --2- 172.21.15.38:0/2391628570 >> v2:172.21.15.104:3300/0 conn(0x7f2f740686a0 0x7f2f74068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f7259c700 1 --2- 172.21.15.38:0/2391628570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f741553e0 0x7f2f740af4d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f640099e0 tx=0x7f2f64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ff5732056e22ab server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f71d9b700 1 -- 172.21.15.38:0/2391628570 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6400b7e0 con 0x7f2f741553e0 2024-08-15T17:58:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f71d9b700 1 -- 172.21.15.38:0/2391628570 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6400b940 con 0x7f2f741553e0 2024-08-15T17:58:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f71d9b700 1 -- 172.21.15.38:0/2391628570 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6400bce0 con 0x7f2f741553e0 2024-08-15T17:58:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f7359e700 1 -- 172.21.15.38:0/2391628570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f741553e0 msgr2=0x7f2f740af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f7359e700 1 --2- 172.21.15.38:0/2391628570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f741553e0 0x7f2f740af4d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f640099e0 tx=0x7f2f64009250 comp rx=0 tx=0).stop 2024-08-15T17:58:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f7359e700 1 -- 172.21.15.38:0/2391628570 shutdown_connections 2024-08-15T17:58:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f7359e700 1 --2- 172.21.15.38:0/2391628570 >> v2:172.21.15.104:3300/0 conn(0x7f2f740686a0 0x7f2f74068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f7359e700 1 --2- 172.21.15.38:0/2391628570 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f741557b0 0x7f2f740b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f7359e700 1 --2- 172.21.15.38:0/2391628570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f741553e0 0x7f2f740af4d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f7359e700 1 -- 172.21.15.38:0/2391628570 >> 172.21.15.38:0/2391628570 conn(0x7f2f7406d800 msgr2=0x7f2f7406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f7359e700 1 -- 172.21.15.38:0/2391628570 shutdown_connections 2024-08-15T17:58:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f7359e700 1 -- 172.21.15.38:0/2391628570 wait complete. 2024-08-15T17:58:49.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.266+0000 7f2f7359e700 1 Processor -- start 2024-08-15T17:58:49.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7359e700 1 -- start start 2024-08-15T17:58:49.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7359e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f740686a0 0x7f2f741480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7359e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f741553e0 0x7f2f740a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7359e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f741557b0 0x7f2f7416a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7359e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f74166dd0 con 0x7f2f740686a0 2024-08-15T17:58:49.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7359e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7416aef0 con 0x7f2f741557b0 2024-08-15T17:58:49.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7359e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f74147c10 con 0x7f2f741553e0 2024-08-15T17:58:49.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f740686a0 0x7f2f741480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f740686a0 0x7f2f741480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34178/0 (socket says 172.21.15.38:34178) 2024-08-15T17:58:49.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7259c700 1 -- 172.21.15.38:0/1346949148 learned_addr learned my addr 172.21.15.38:0/1346949148 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f6bfff700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f741553e0 0x7f2f740a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f72d9d700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f741557b0 0x7f2f7416a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7259c700 1 -- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f741553e0 msgr2=0x7f2f740a7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7259c700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f741553e0 0x7f2f740a7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7259c700 1 -- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f741557b0 msgr2=0x7f2f7416a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7259c700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f741557b0 0x7f2f7416a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7259c700 1 -- 172.21.15.38:0/1346949148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f640094f0 con 0x7f2f740686a0 2024-08-15T17:58:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7259c700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f740686a0 0x7f2f741480a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009290 tx=0x7f2f64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f6a7fc700 1 -- 172.21.15.38:0/1346949148 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64011d30 con 0x7f2f740686a0 2024-08-15T17:58:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f6a7fc700 1 -- 172.21.15.38:0/1346949148 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6401f070 con 0x7f2f740686a0 2024-08-15T17:58:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7359e700 1 -- 172.21.15.38:0/1346949148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7416ac10 con 0x7f2f740686a0 2024-08-15T17:58:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f6a7fc700 1 -- 172.21.15.38:0/1346949148 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f640266d0 con 0x7f2f740686a0 2024-08-15T17:58:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.270+0000 7f2f7359e700 1 -- 172.21.15.38:0/1346949148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f741478a0 con 0x7f2f740686a0 2024-08-15T17:58:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.274+0000 7f2f7359e700 1 -- 172.21.15.38:0/1346949148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f740638f0 con 0x7f2f740686a0 2024-08-15T17:58:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.274+0000 7f2f6a7fc700 1 -- 172.21.15.38:0/1346949148 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2f6401a430 con 0x7f2f740686a0 2024-08-15T17:58:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.274+0000 7f2f6a7fc700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f54060860 0x7f2f54062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.274+0000 7f2f6a7fc700 1 -- 172.21.15.38:0/1346949148 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f64021070 con 0x7f2f740686a0 2024-08-15T17:58:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.274+0000 7f2f6bfff700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f54060860 0x7f2f54062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.274+0000 7f2f6bfff700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f54060860 0x7f2f54062d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2f7414cf80 tx=0x7f2f5c0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:49.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.282+0000 7f2f6a7fc700 1 -- 172.21.15.38:0/1346949148 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2f6404ed90 con 0x7f2f740686a0 2024-08-15T17:58:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.482+0000 7f2f7359e700 1 -- 172.21.15.38:0/1346949148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2f740cb7d0 con 0x7f2f740686a0 2024-08-15T17:58:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f6a7fc700 1 -- 172.21.15.38:0/1346949148 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2f64015070 con 0x7f2f740686a0 2024-08-15T17:58:49.487 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:58:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 -- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f54060860 msgr2=0x7f2f54062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f54060860 0x7f2f54062d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2f7414cf80 tx=0x7f2f5c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 -- 172.21.15.38:0/1346949148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f740686a0 msgr2=0x7f2f741480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:49.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f740686a0 0x7f2f741480a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009290 tx=0x7f2f64017000 comp rx=0 tx=0).stop 2024-08-15T17:58:49.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 -- 172.21.15.38:0/1346949148 shutdown_connections 2024-08-15T17:58:49.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f54060860 0x7f2f54062d10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f741557b0 0x7f2f7416a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f741553e0 0x7f2f740a7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 --2- 172.21.15.38:0/1346949148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f740686a0 0x7f2f741480a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:49.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 -- 172.21.15.38:0/1346949148 >> 172.21.15.38:0/1346949148 conn(0x7f2f7406d800 msgr2=0x7f2f7406dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:49.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 -- 172.21.15.38:0/1346949148 shutdown_connections 2024-08-15T17:58:49.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:49.486+0000 7f2f7359e700 1 -- 172.21.15.38:0/1346949148 wait complete. 2024-08-15T17:58:49.501 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:58:49.501 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:58:49.501 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-08-15T17:58:49.501 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-15T17:58:49.501 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-15T17:58:49.501 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:58:49.501 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:58:49.501 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:58:49.501 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:58:49.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.502 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-08-15T17:58:49.509 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42616sec preferred_lft 42616sec 2024-08-15T17:58:49.510 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-08-15T17:58:49.511 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:49.511 INFO:teuthology.orchestra.run.smithi038.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:49.511 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:49.511 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:49.511 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:49.511 INFO:teuthology.orchestra.run.smithi038.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:58:49.511 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 02:42:40:33:56:ee brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:49.511 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:58:49.511 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:49.511 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-15T17:58:49.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.512 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-08-15T17:58:49.512 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-08-15T17:58:49.512 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-08-15T17:58:49.512 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-08-15T17:58:49.512 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-08-15T17:58:49.512 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-15T17:58:49.512 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-08-15T17:58:49.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.554 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:58:49.613 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-08-15T17:58:49.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.615 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-08-15T17:58:49.632 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-08-15T17:58:49.633 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:58:49.633 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-08-15T17:58:49.633 INFO:teuthology.orchestra.run.smithi038.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:58:49.633 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-08-15T17:58:49.633 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:58:49.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.634 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-08-15T17:58:49.634 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-08-15T17:58:49.634 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-15T17:58:49.634 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-15T17:58:49.634 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-15T17:58:49.634 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-08-15T17:58:49.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.676 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-08-15T17:58:49.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.683 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-08-15T17:58:49.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.689 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-08-15T17:58:49.689 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-08-15T17:58:49.689 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T17:58:49.690 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-15T17:58:49.690 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-08-15T17:58:49.716 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-15T17:58:49.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.716 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-08-15T17:58:49.716 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-08-15T17:58:49.716 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-15T17:58:49.716 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-15T17:58:49.716 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-15T17:58:49.716 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-15T17:58:49.717 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-15T17:58:49.717 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-08-15T17:58:49.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:49.844 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-08-15T17:58:49.845 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-08-15T17:58:49.845 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-08-15T17:58:49.845 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-08-15T17:58:49.845 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-08-15T17:58:49.872 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-08-15T17:58:49.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:58:49.872 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-15T17:58:49.878 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:58:49.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:58:49.878 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-15T17:58:49.883 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-08-15T17:58:49.942 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:49.942 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:49.954 INFO:teuthology.orchestra.run.smithi038.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:49.955 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:49.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:49.956 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-08-15T17:58:49.960 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T17:58:49.961 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-08-15T17:58:50.008 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-08-15T17:58:50.027 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:50.027 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:50.038 INFO:teuthology.orchestra.run.smithi038.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:50.039 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:50.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:50.040 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-08-15T17:58:50.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-08-15T17:58:50.090+0000 7f06c77f2380 -1 init, newargv = 0x562d56e0fa20 newargc=15 2024-08-15T17:58:50.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[31024]: starting ceph client 2024-08-15T17:58:50.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[31024]: starting fuse 2024-08-15T17:58:51.146 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-08-15T17:58:51.163 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:51.163 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:51.222 INFO:teuthology.orchestra.run.smithi038.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:51.223 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:51.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:51.224 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-08-15T17:58:51.234 INFO:teuthology.orchestra.run.smithi038.stdout:52 2024-08-15T17:58:51.234 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-15T17:58:51.234 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:58:51.234 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-08-15T17:58:51.234 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-08-15T17:58:51.235 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:58:51.236 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:58:51.236 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:58:51.236 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-08-15T17:58:51.236 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-08-15T17:58:51.236 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-08-15T17:58:51.236 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:58:51.236 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:58:51.236 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-08-15T17:58:51.236 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-08-15T17:58:51.322 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.31024.asok 2024-08-15T17:58:51.507 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.31024.asok 2024-08-15T17:58:51.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:51.507 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.31024.asok status 2024-08-15T17:58:51.649 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T17:58:51.649 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-08-15T17:58:51.649 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:58:51.649 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:58:51.649 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-08-15T17:58:51.649 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-08-15T17:58:51.649 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-08-15T17:58:51.649 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "31024", 2024-08-15T17:58:51.649 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-08-15T17:58:51.650 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:58:51.650 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-08-15T17:58:51.650 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-08-15T17:58:51.650 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4689, 2024-08-15T17:58:51.650 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-08-15T17:58:51.650 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-08-15T17:58:51.650 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-08-15T17:58:51.650 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 4689 2024-08-15T17:58:51.650 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:58:51.650 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 211557003 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 211557003 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.4689 172.21.15.38:0/211557003", 2024-08-15T17:58:51.651 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "172.21.15.38:0/211557003", 2024-08-15T17:58:51.652 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-08-15T17:58:51.652 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 7, 2024-08-15T17:58:51.652 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 21, 2024-08-15T17:58:51.652 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:58:51.652 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-08-15T17:58:51.652 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-08-15T17:58:51.652 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T17:58:51.658 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:58:51.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf1797700 1 Processor -- start 2024-08-15T17:58:51.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf1797700 1 -- start start 2024-08-15T17:58:51.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf1797700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbdec0ac2a0 0x7fbdec0adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf1797700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0aac40 0x7fbdec162c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf1797700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdec0aeac0 con 0x7fbdec0ae160 2024-08-15T17:58:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf1797700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdec0ae5e0 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf1797700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0af620 con 0x7fbdec0aac40 2024-08-15T17:58:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf0f96700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0aac40 0x7fbdec162c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf0f96700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0aac40 0x7fbdec162c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:36002/0 (socket says 172.21.15.38:36002) 2024-08-15T17:58:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf0f96700 1 -- 172.21.15.38:0/1858016212 learned_addr learned my addr 172.21.15.38:0/1858016212 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdeb7fe700 1 --1- 172.21.15.38:0/1858016212 >> v1:172.21.15.104:6789/0 conn(0x7fbdec0ae160 0x7fbdec0aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:38250/0 (socket says 172.21.15.38:38250) 2024-08-15T17:58:51.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdebfff700 1 --2- 172.21.15.38:0/1858016212 >> v2:172.21.15.104:3300/0 conn(0x7fbdec0ac2a0 0x7fbdec0adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdeaffd700 1 -- 172.21.15.38:0/1858016212 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520950863 0 0) 0x7fbdec0aeac0 con 0x7fbdec0ae160 2024-08-15T17:58:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdeaffd700 1 -- 172.21.15.38:0/1858016212 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd8003570 con 0x7fbdec0ae160 2024-08-15T17:58:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf0f96700 1 -- 172.21.15.38:0/1858016212 >> v2:172.21.15.104:3300/0 conn(0x7fbdec0ac2a0 msgr2=0x7fbdec0adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf0f96700 1 --2- 172.21.15.38:0/1858016212 >> v2:172.21.15.104:3300/0 conn(0x7fbdec0ac2a0 0x7fbdec0adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf0f96700 1 -- 172.21.15.38:0/1858016212 >> v1:172.21.15.104:6789/0 conn(0x7fbdec0ae160 legacy=0x7fbdec0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdf0f96700 1 -- 172.21.15.38:0/1858016212 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec0b1eb0 con 0x7fbdec0aac40 2024-08-15T17:58:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.762+0000 7fbdebfff700 1 --2- 172.21.15.38:0/1858016212 >> v2:172.21.15.104:3300/0 conn(0x7fbdec0ac2a0 0x7fbdec0adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:58:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdf0f96700 1 --2- 172.21.15.38:0/1858016212 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0aac40 0x7fbdec162c80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009f20 tx=0x7fbde4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e36c8cda4988aba server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdeaffd700 1 -- 172.21.15.38:0/1858016212 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde400e070 con 0x7fbdec0aac40 2024-08-15T17:58:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdeaffd700 1 -- 172.21.15.38:0/1858016212 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde400b970 con 0x7fbdec0aac40 2024-08-15T17:58:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdeaffd700 1 -- 172.21.15.38:0/1858016212 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde40092e0 con 0x7fbdec0aac40 2024-08-15T17:58:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdf1797700 1 -- 172.21.15.38:0/1858016212 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0aac40 msgr2=0x7fbdec162c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdf1797700 1 --2- 172.21.15.38:0/1858016212 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0aac40 0x7fbdec162c80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009f20 tx=0x7fbde4009840 comp rx=0 tx=0).stop 2024-08-15T17:58:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdf1797700 1 -- 172.21.15.38:0/1858016212 shutdown_connections 2024-08-15T17:58:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdf1797700 1 --2- 172.21.15.38:0/1858016212 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0aac40 0x7fbdec162c80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdf1797700 1 --2- 172.21.15.38:0/1858016212 >> v2:172.21.15.104:3300/0 conn(0x7fbdec0ac2a0 0x7fbdec0adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdf1797700 1 -- 172.21.15.38:0/1858016212 >> 172.21.15.38:0/1858016212 conn(0x7fbdec068320 msgr2=0x7fbdec0a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdf1797700 1 -- 172.21.15.38:0/1858016212 shutdown_connections 2024-08-15T17:58:51.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdf1797700 1 -- 172.21.15.38:0/1858016212 wait complete. 2024-08-15T17:58:51.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdf1797700 1 Processor -- start 2024-08-15T17:58:51.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.766+0000 7fbdf1797700 1 -- start start 2024-08-15T17:58:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdf1797700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec0aac40 0x7fbdec154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdf1797700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdec0ac2a0 0x7fbdec153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdf1797700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0ae160 0x7fbdec15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdf1797700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0c6390 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdf1797700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec16c7e0 con 0x7fbdec0aac40 2024-08-15T17:58:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdf1797700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec16cac0 con 0x7fbdec0ae160 2024-08-15T17:58:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdec0ac2a0 0x7fbdec153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdec0ac2a0 0x7fbdec153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34186/0 (socket says 172.21.15.38:34186) 2024-08-15T17:58:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdf0f96700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0ae160 0x7fbdec15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdeb7fe700 1 -- 172.21.15.38:0/429918965 learned_addr learned my addr 172.21.15.38:0/429918965 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdebfff700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec0aac40 0x7fbdec154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdeb7fe700 1 -- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0ae160 msgr2=0x7fbdec15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdeb7fe700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0ae160 0x7fbdec15e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdeb7fe700 1 -- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec0aac40 msgr2=0x7fbdec154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdeb7fe700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec0aac40 0x7fbdec154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdeb7fe700 1 -- 172.21.15.38:0/429918965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd4009420 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdeb7fe700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdec0ac2a0 0x7fbdec153c50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00c590 tx=0x7fbddc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbde3fff700 1 -- 172.21.15.38:0/429918965 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc005a70 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbde3fff700 1 -- 172.21.15.38:0/429918965 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc005bf0 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdf1797700 1 -- 172.21.15.38:0/429918965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde4009ae0 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbde3fff700 1 -- 172.21.15.38:0/429918965 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0070d0 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdf1797700 1 -- 172.21.15.38:0/429918965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdec154cd0 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbdf1797700 1 -- 172.21.15.38:0/429918965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdec0643a0 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbde3fff700 1 -- 172.21.15.38:0/429918965 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbddc0074c0 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.770+0000 7fbde3fff700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbdcc060860 0x7fbdcc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.774+0000 7fbde3fff700 1 -- 172.21.15.38:0/429918965 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbddc07bfd0 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.774+0000 7fbdebfff700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbdcc060860 0x7fbdcc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.774+0000 7fbdebfff700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbdcc060860 0x7fbdcc062d10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009f40 tx=0x7fbdd40091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:51.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.778+0000 7fbde3fff700 1 -- 172.21.15.38:0/429918965 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbddc04c420 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 -- 172.21.15.38:0/429918965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbdec0aa490 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbde3fff700 1 -- 172.21.15.38:0/429918965 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbddc017060 con 0x7fbdec0ac2a0 2024-08-15T17:58:51.976 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:58:51.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 -- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbdcc060860 msgr2=0x7fbdcc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:51.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbdcc060860 0x7fbdcc062d10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009f40 tx=0x7fbdd40091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:51.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 -- 172.21.15.38:0/429918965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdec0ac2a0 msgr2=0x7fbdec153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:51.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdec0ac2a0 0x7fbdec153c50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00c590 tx=0x7fbddc007e00 comp rx=0 tx=0).stop 2024-08-15T17:58:51.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 -- 172.21.15.38:0/429918965 shutdown_connections 2024-08-15T17:58:51.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbdcc060860 0x7fbdcc062d10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:51.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbdec0ae160 0x7fbdec15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:51.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdec0ac2a0 0x7fbdec153c50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:51.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 --2- 172.21.15.38:0/429918965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec0aac40 0x7fbdec154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:51.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 -- 172.21.15.38:0/429918965 >> 172.21.15.38:0/429918965 conn(0x7fbdec068320 msgr2=0x7fbdec069430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:51.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.974+0000 7fbdf1797700 1 -- 172.21.15.38:0/429918965 shutdown_connections 2024-08-15T17:58:51.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:51.978+0000 7fbdf1797700 1 -- 172.21.15.38:0/429918965 wait complete. 2024-08-15T17:58:51.989 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:58:51.989 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:58:51.989 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-08-15T17:58:51.989 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-08-15T17:58:51.989 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-08-15T17:58:51.989 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:58:51.989 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:58:51.990 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:58:51.990 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-08-15T17:58:51.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:51.990 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-08-15T17:58:51.999 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:58:51.999 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:58:51.999 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:58:51.999 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.000 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-08-15T17:58:52.000 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.000 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:52.000 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:52.000 INFO:teuthology.orchestra.run.smithi038.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:58:52.000 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:52.001 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-08-15T17:58:52.001 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42614sec preferred_lft 42614sec 2024-08-15T17:58:52.001 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-08-15T17:58:52.001 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.001 INFO:teuthology.orchestra.run.smithi038.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:52.001 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:52.002 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:52.002 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:52.003 INFO:teuthology.orchestra.run.smithi038.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:58:52.003 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 02:42:40:33:56:ee brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:52.003 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:58:52.003 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.003 INFO:teuthology.orchestra.run.smithi038.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:58:52.003 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether b6:52:e5:4f:e7:cd brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:52.003 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:58:52.003 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.003 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::2cd6:e8ff:fe2f:5f2a/64 scope link 2024-08-15T17:58:52.004 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.004 INFO:teuthology.orchestra.run.smithi038.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:58:52.004 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether b6:52:e5:4f:e7:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T17:58:52.004 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::b452:e5ff:fe4f:e7cd/64 scope link 2024-08-15T17:58:52.004 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:52.004 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:58:52.056 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-08-15T17:58:52.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:52.058 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-08-15T17:58:52.065 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-08-15T17:58:52.065 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:58:52.065 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-08-15T17:58:52.065 INFO:teuthology.orchestra.run.smithi038.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:58:52.065 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-08-15T17:58:52.065 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:58:52.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:52.066 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-08-15T17:58:52.066 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-08-15T17:58:52.066 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-15T17:58:52.066 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-15T17:58:52.066 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-15T17:58:52.066 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-08-15T17:58:52.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:52.104 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-08-15T17:58:52.153 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-15T17:58:52.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:52.153 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-08-15T17:58:52.161 INFO:teuthology.orchestra.run.smithi038.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-08-15T17:58:52.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:52.161 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-08-15T17:58:52.162 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-08-15T17:58:52.162 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-15T17:58:52.162 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-08-15T17:58:52.162 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-08-15T17:58:52.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:52.190 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-08-15T17:58:52.248 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:58:52.248 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:58:52.248 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:58:52.248 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.249 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-08-15T17:58:52.249 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.249 INFO:teuthology.orchestra.run.smithi038.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:58:52.249 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:ea:14:de:d1:a7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:58:52.249 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:58:52.249 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.249 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38ea:14ff:fede:d1a7/64 scope link 2024-08-15T17:58:52.249 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:52.250 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-08-15T17:58:52.262 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:58:52.262 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:58:52.262 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:58:52.262 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.263 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-08-15T17:58:52.263 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.263 INFO:teuthology.orchestra.run.smithi038.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:58:52.263 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:ea:14:de:d1:a7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:58:52.263 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:58:52.263 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.263 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38ea:14ff:fede:d1a7/64 scope link 2024-08-15T17:58:52.263 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:52.264 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-08-15T17:58:52.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:52.264 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-08-15T17:58:52.264 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-08-15T17:58:52.265 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-08-15T17:58:52.265 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-08-15T17:58:52.265 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-08-15T17:58:52.265 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-08-15T17:58:52.265 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-08-15T17:58:52.265 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-08-15T17:58:52.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:52.318 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-08-15T17:58:52.319 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-08-15T17:58:52.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.1 up 2024-08-15T17:58:52.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.1 master ceph-brx 2024-08-15T17:58:52.319 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-08-15T17:58:52.343 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-08-15T17:58:52.343 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:58:52.343 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-08-15T17:58:52.350 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-08-15T17:58:52.351 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:58:52.351 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-08-15T17:58:52.357 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-08-15T17:58:52.420 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:52.420 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:52.430 INFO:teuthology.orchestra.run.smithi038.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:52.431 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:52.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:52.432 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-08-15T17:58:52.439 INFO:teuthology.orchestra.run.smithi038.stdout:52 2024-08-15T17:58:52.439 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-08-15T17:58:52.439 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-08-15T17:58:52.442 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-08-15T17:58:52.459 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:52.459 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:52.516 INFO:teuthology.orchestra.run.smithi038.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:52.517 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:52.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:52.519 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-08-15T17:58:52.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi038.stderr:2024-08-15T17:58:52.522+0000 7f9099d2d380 -1 init, newargv = 0x563c05d54b90 newargc=15 2024-08-15T17:58:52.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi038.stderr:ceph-fuse[31162]: starting ceph client 2024-08-15T17:58:52.526 INFO:teuthology.orchestra.run.smithi038.stdout:52 2024-08-15T17:58:52.533 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi038.stderr:ceph-fuse[31162]: starting fuse 2024-08-15T17:58:53.627 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-08-15T17:58:53.646 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:53.646 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:53.706 INFO:teuthology.orchestra.run.smithi038.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:53.707 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:53.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:53.707 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-08-15T17:58:53.715 INFO:teuthology.orchestra.run.smithi038.stdout:52 2024-08-15T17:58:53.715 INFO:teuthology.orchestra.run.smithi038.stdout:53 2024-08-15T17:58:53.715 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-08-15T17:58:53.715 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:58:53.715 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-08-15T17:58:53.715 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-08-15T17:58:53.715 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-08-15T17:58:53.715 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-08-15T17:58:53.716 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:58:53.716 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-08-15T17:58:53.716 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-08-15T17:58:53.716 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-08-15T17:58:53.717 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-08-15T17:58:53.718 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:58:53.718 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:58:53.718 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.2")) 2024-08-15T17:58:53.718 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-08-15T17:58:53.802 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.2.31162.asok 2024-08-15T17:58:53.985 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.31162.asok 2024-08-15T17:58:53.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:53.985 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.31162.asok status 2024-08-15T17:58:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T17:58:54.123 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-08-15T17:58:54.123 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:58:54.123 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "2", 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "31162", 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4698, 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 4698 2024-08-15T17:58:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3705856415 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3705856415 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.4698 192.168.0.2:0/3705856415", 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.2:0/3705856415", 2024-08-15T17:58:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-08-15T17:58:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 7, 2024-08-15T17:58:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 21, 2024-08-15T17:58:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:58:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-08-15T17:58:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-08-15T17:58:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T17:58:54.133 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:58:54.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.242+0000 7f6c9206a700 1 Processor -- start 2024-08-15T17:58:54.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.242+0000 7f6c9206a700 1 -- start start 2024-08-15T17:58:54.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.242+0000 7f6c9206a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c8c06e9d0 0x7f6c8c06eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:54.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.242+0000 7f6c9206a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6c8c0ab540 0x7f6c8c0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:54.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.242+0000 7f6c9206a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6c8c06df10 0x7f6c8c155010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:54.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.242+0000 7f6c9206a700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8c0aba70 con 0x7f6c8c0ab540 2024-08-15T17:58:54.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.242+0000 7f6c9206a700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8c0abd50 con 0x7f6c8c06df10 2024-08-15T17:58:54.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.242+0000 7f6c9206a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c0ac030 con 0x7f6c8c06e9d0 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c90867700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6c8c0ab540 0x7f6c8c0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c90867700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6c8c0ab540 0x7f6c8c0b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34194/0 (socket says 172.21.15.38:34194) 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c90867700 1 -- 172.21.15.38:0/2908202978 learned_addr learned my addr 172.21.15.38:0/2908202978 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c91869700 1 --2- 172.21.15.38:0/2908202978 >> v2:172.21.15.104:3300/0 conn(0x7f6c8c06df10 0x7f6c8c155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c91068700 1 --2- 172.21.15.38:0/2908202978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c8c06e9d0 0x7f6c8c06eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c90867700 1 -- 172.21.15.38:0/2908202978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c8c06e9d0 msgr2=0x7f6c8c06eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c90867700 1 --2- 172.21.15.38:0/2908202978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c8c06e9d0 0x7f6c8c06eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c90867700 1 -- 172.21.15.38:0/2908202978 >> v2:172.21.15.104:3300/0 conn(0x7f6c8c06df10 msgr2=0x7f6c8c155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c90867700 1 --2- 172.21.15.38:0/2908202978 >> v2:172.21.15.104:3300/0 conn(0x7f6c8c06df10 0x7f6c8c155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c90867700 1 -- 172.21.15.38:0/2908202978 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c0ac310 con 0x7f6c8c0ab540 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c91068700 1 --2- 172.21.15.38:0/2908202978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c8c06e9d0 0x7f6c8c06eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c90867700 1 --2- 172.21.15.38:0/2908202978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c8c0ab540 0x7f6c8c0b1320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0ae820 tx=0x7f6c7c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b332d85a516b9eaa server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c83fff700 1 -- 172.21.15.38:0/2908202978 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c00dba0 con 0x7f6c8c0ab540 2024-08-15T17:58:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c83fff700 1 -- 172.21.15.38:0/2908202978 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c00dd00 con 0x7f6c8c0ab540 2024-08-15T17:58:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c83fff700 1 -- 172.21.15.38:0/2908202978 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c0196d0 con 0x7f6c8c0ab540 2024-08-15T17:58:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c9206a700 1 -- 172.21.15.38:0/2908202978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c8c0ab540 msgr2=0x7f6c8c0b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c9206a700 1 --2- 172.21.15.38:0/2908202978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c8c0ab540 0x7f6c8c0b1320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0ae820 tx=0x7f6c7c00b4f0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c9206a700 1 -- 172.21.15.38:0/2908202978 shutdown_connections 2024-08-15T17:58:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c9206a700 1 --2- 172.21.15.38:0/2908202978 >> v2:172.21.15.104:3300/0 conn(0x7f6c8c06df10 0x7f6c8c155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c9206a700 1 --2- 172.21.15.38:0/2908202978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c8c0ab540 0x7f6c8c0b1320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c9206a700 1 --2- 172.21.15.38:0/2908202978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c8c06e9d0 0x7f6c8c06eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c9206a700 1 -- 172.21.15.38:0/2908202978 >> 172.21.15.38:0/2908202978 conn(0x7f6c8c0678d0 msgr2=0x7f6c8c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:54.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c9206a700 1 -- 172.21.15.38:0/2908202978 shutdown_connections 2024-08-15T17:58:54.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c9206a700 1 -- 172.21.15.38:0/2908202978 wait complete. 2024-08-15T17:58:54.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c9206a700 1 Processor -- start 2024-08-15T17:58:54.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.246+0000 7f6c9206a700 1 -- start start 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c9206a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c8c06df10 0x7f6c8c16c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c9206a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c8c06e9d0 0x7f6c8c1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c9206a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c8c0ab540 0x7f6c8c159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c9206a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c1698a0 con 0x7f6c8c0ab540 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c9206a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c14c310 con 0x7f6c8c06df10 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c9206a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c14b700 con 0x7f6c8c06e9d0 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c91869700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c8c0ab540 0x7f6c8c159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c91869700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c8c0ab540 0x7f6c8c159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34210/0 (socket says 172.21.15.38:34210) 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c91869700 1 -- 172.21.15.38:0/4007559462 learned_addr learned my addr 172.21.15.38:0/4007559462 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c90867700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c8c06e9d0 0x7f6c8c1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c91068700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c8c06df10 0x7f6c8c16c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c91869700 1 -- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c8c06e9d0 msgr2=0x7f6c8c1591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:54.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c91869700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c8c06e9d0 0x7f6c8c1591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c91869700 1 -- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c8c06df10 msgr2=0x7f6c8c16c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c91869700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c8c06df10 0x7f6c8c16c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c91869700 1 -- 172.21.15.38:0/4007559462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c00b790 con 0x7f6c8c0ab540 2024-08-15T17:58:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c91068700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c8c06df10 0x7f6c8c16c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c91869700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c8c0ab540 0x7f6c8c159730 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009e90 tx=0x7f6c8400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c817fa700 1 -- 172.21.15.38:0/4007559462 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400cc20 con 0x7f6c8c0ab540 2024-08-15T17:58:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c817fa700 1 -- 172.21.15.38:0/4007559462 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8400f040 con 0x7f6c8c0ab540 2024-08-15T17:58:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c9206a700 1 -- 172.21.15.38:0/4007559462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8c14b9e0 con 0x7f6c8c0ab540 2024-08-15T17:58:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c817fa700 1 -- 172.21.15.38:0/4007559462 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84014480 con 0x7f6c8c0ab540 2024-08-15T17:58:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c9206a700 1 -- 172.21.15.38:0/4007559462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c14bfa0 con 0x7f6c8c0ab540 2024-08-15T17:58:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c817fa700 1 -- 172.21.15.38:0/4007559462 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6c84014830 con 0x7f6c8c0ab540 2024-08-15T17:58:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.250+0000 7f6c9206a700 1 -- 172.21.15.38:0/4007559462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8c14c480 con 0x7f6c8c0ab540 2024-08-15T17:58:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.254+0000 7f6c817fa700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6c70060860 0x7f6c70062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.254+0000 7f6c817fa700 1 -- 172.21.15.38:0/4007559462 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6c8407d600 con 0x7f6c8c0ab540 2024-08-15T17:58:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.254+0000 7f6c91068700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6c70060860 0x7f6c70062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:54.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.254+0000 7f6c91068700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6c70060860 0x7f6c70062d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c88006f00 tx=0x7f6c88008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:54.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.262+0000 7f6c817fa700 1 -- 172.21.15.38:0/4007559462 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6c840514e0 con 0x7f6c8c0ab540 2024-08-15T17:58:54.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.462+0000 7f6c9206a700 1 -- 172.21.15.38:0/4007559462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6c8c0643a0 con 0x7f6c8c0ab540 2024-08-15T17:58:54.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.462+0000 7f6c817fa700 1 -- 172.21.15.38:0/4007559462 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6c84051070 con 0x7f6c8c0ab540 2024-08-15T17:58:54.465 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:58:54.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 -- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6c70060860 msgr2=0x7f6c70062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:54.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6c70060860 0x7f6c70062d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c88006f00 tx=0x7f6c88008040 comp rx=0 tx=0).stop 2024-08-15T17:58:54.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 -- 172.21.15.38:0/4007559462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c8c0ab540 msgr2=0x7f6c8c159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:54.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c8c0ab540 0x7f6c8c159730 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009e90 tx=0x7f6c8400b040 comp rx=0 tx=0).stop 2024-08-15T17:58:54.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 -- 172.21.15.38:0/4007559462 shutdown_connections 2024-08-15T17:58:54.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6c70060860 0x7f6c70062d10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c8c0ab540 0x7f6c8c159730 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c8c06e9d0 0x7f6c8c1591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 --2- 172.21.15.38:0/4007559462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c8c06df10 0x7f6c8c16c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:54.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 -- 172.21.15.38:0/4007559462 >> 172.21.15.38:0/4007559462 conn(0x7f6c8c0678d0 msgr2=0x7f6c8c1516d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:54.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 -- 172.21.15.38:0/4007559462 shutdown_connections 2024-08-15T17:58:54.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:54.466+0000 7f6c9206a700 1 -- 172.21.15.38:0/4007559462 wait complete. 2024-08-15T17:58:54.479 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:58:54.479 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:58:54.479 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-08-15T17:58:54.479 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-08-15T17:58:54.479 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-08-15T17:58:54.479 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:58:54.479 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:58:54.479 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:58:54.480 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:58:54.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:54.480 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-08-15T17:58:54.488 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:58:54.488 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:58:54.488 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:58:54.489 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42603sec preferred_lft 42603sec 2024-08-15T17:58:54.490 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-08-15T17:58:54.490 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:54.490 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:58:54.490 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:51:73:f0:c7 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:54.490 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:58:54.490 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:54.490 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-15T17:58:54.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:54.490 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2024-08-15T17:58:54.491 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-08-15T17:58:54.491 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add name ceph-brx type bridge 2024-08-15T17:58:54.491 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip addr flush dev ceph-brx 2024-08-15T17:58:54.491 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set ceph-brx up 2024-08-15T17:58:54.491 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-15T17:58:54.491 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-08-15T17:58:54.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:54.574 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:58:54.584 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-08-15T17:58:54.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:54.586 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-08-15T17:58:54.611 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-08-15T17:58:54.611 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:58:54.611 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:58:54.611 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:58:54.611 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:58:54.612 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:58:54.612 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:58:54.612 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:58:54.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:54.613 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2024-08-15T17:58:54.613 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-08-15T17:58:54.613 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:58:54.613 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:58:54.613 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:58:54.613 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-08-15T17:58:54.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:54.696 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-08-15T17:58:54.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:54.704 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list-id 2024-08-15T17:58:54.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:54.711 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2024-08-15T17:58:54.711 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-08-15T17:58:54.711 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T17:58:54.712 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-08-15T17:58:54.712 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-08-15T17:58:54.742 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-08-15T17:58:54.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:54.742 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2024-08-15T17:58:54.742 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-08-15T17:58:54.742 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-08-15T17:58:54.742 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-15T17:58:54.742 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-08-15T17:58:54.742 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-08-15T17:58:54.742 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-08-15T17:58:54.743 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-08-15T17:58:54.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:54.824 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2024-08-15T17:58:54.824 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-08-15T17:58:54.824 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-08-15T17:58:54.824 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set dev brx.0 master ceph-brx 2024-08-15T17:58:54.824 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-08-15T17:58:54.847 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-08-15T17:58:54.848 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:58:54.848 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-08-15T17:58:54.854 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:58:54.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:58:54.854 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:58:54.860 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2024-08-15T17:58:54.872 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:54.873 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:54.885 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:54.886 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:54.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:54.886 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2024-08-15T17:58:54.892 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T17:58:54.892 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-08-15T17:58:54.894 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2024-08-15T17:58:54.912 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:54.912 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:54.969 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:54.970 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:54.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:54.971 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2024-08-15T17:58:54.977 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi104.stderr:2024-08-15T17:58:54.973+0000 7f00fd398380 -1 init, newargv = 0x55ea36c2a8c0 newargc=15 2024-08-15T17:58:54.977 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi104.stderr:ceph-fuse[29941]: starting ceph client 2024-08-15T17:58:54.985 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi104.stderr:ceph-fuse[29941]: starting fuse 2024-08-15T17:58:56.077 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2024-08-15T17:58:56.096 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:56.097 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:56.107 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:56.109 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:56.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:56.109 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2024-08-15T17:58:56.160 INFO:teuthology.orchestra.run.smithi104.stdout:52 2024-08-15T17:58:56.160 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-15T17:58:56.160 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:58:56.160 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> import re 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> import os 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:58:56.161 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2024-08-15T17:58:56.162 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:58:56.162 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:58:56.162 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:58:56.162 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2024-08-15T17:58:56.162 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2024-08-15T17:58:56.162 DEBUG:teuthology.orchestra.run.smithi104:> return f 2024-08-15T17:58:56.162 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:58:56.162 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:58:56.162 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.1")) 2024-08-15T17:58:56.162 DEBUG:teuthology.orchestra.run.smithi104:> ' 2024-08-15T17:58:56.247 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.1.29941.asok 2024-08-15T17:58:56.432 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.29941.asok 2024-08-15T17:58:56.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:56.432 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.29941.asok status 2024-08-15T17:58:56.523 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "1", 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "29941", 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2024-08-15T17:58:56.524 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 4553, 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 4553 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 3582889672 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2024-08-15T17:58:56.525 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 3582889672 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.4553 192.168.0.1:0/3582889672", 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "192.168.0.1:0/3582889672", 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 7, 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 21, 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2024-08-15T17:58:56.526 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-08-15T17:58:56.533 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:58:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.645+0000 7efc39672700 1 Processor -- start 2024-08-15T17:58:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.645+0000 7efc39672700 1 -- start start 2024-08-15T17:58:56.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.645+0000 7efc39672700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efc340af3f0 0x7efc34068c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:56.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.645+0000 7efc39672700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc340b05e0 con 0x7efc340b0a70 2024-08-15T17:58:56.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.645+0000 7efc39672700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc340af9d0 con 0x7efc34068840 2024-08-15T17:58:56.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.645+0000 7efc39672700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc340afcb0 con 0x7efc340af3f0 2024-08-15T17:58:56.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.645+0000 7efc38e71700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7efc340b0a70 0x7efc34155810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41002/0 (socket says 172.21.15.38:41002) 2024-08-15T17:58:56.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.645+0000 7efc38e71700 1 -- 172.21.15.38:0/2772879088 learned_addr learned my addr 172.21.15.38:0/2772879088 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:56.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.645+0000 7efc337fe700 1 --2- 172.21.15.38:0/2772879088 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efc340af3f0 0x7efc34068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:56.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.645+0000 7efc32ffd700 1 -- 172.21.15.38:0/2772879088 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107251482 0 0) 0x7efc340b05e0 con 0x7efc340b0a70 2024-08-15T17:58:56.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc32ffd700 1 -- 172.21.15.38:0/2772879088 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc1c003570 con 0x7efc340b0a70 2024-08-15T17:58:56.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc32ffd700 1 -- 172.21.15.38:0/2772879088 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039957835 0 0) 0x7efc340af9d0 con 0x7efc34068840 2024-08-15T17:58:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc32ffd700 1 -- 172.21.15.38:0/2772879088 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc340b05e0 con 0x7efc34068840 2024-08-15T17:58:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc337fe700 1 -- 172.21.15.38:0/2772879088 >> v1:172.21.15.104:6789/0 conn(0x7efc34068840 legacy=0x7efc340ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc337fe700 1 -- 172.21.15.38:0/2772879088 >> v1:172.21.15.38:6789/0 conn(0x7efc340b0a70 legacy=0x7efc34155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc337fe700 1 -- 172.21.15.38:0/2772879088 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc340aff90 con 0x7efc340af3f0 2024-08-15T17:58:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc337fe700 1 --2- 172.21.15.38:0/2772879088 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efc340af3f0 0x7efc34068c10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efc28009910 tx=0x7efc280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff8103a8e02f271 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc32ffd700 1 -- 172.21.15.38:0/2772879088 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28011630 con 0x7efc340af3f0 2024-08-15T17:58:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc32ffd700 1 -- 172.21.15.38:0/2772879088 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc28011790 con 0x7efc340af3f0 2024-08-15T17:58:56.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc32ffd700 1 -- 172.21.15.38:0/2772879088 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28011b30 con 0x7efc340af3f0 2024-08-15T17:58:56.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc39672700 1 -- 172.21.15.38:0/2772879088 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efc340af3f0 msgr2=0x7efc34068c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:56.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc39672700 1 --2- 172.21.15.38:0/2772879088 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efc340af3f0 0x7efc34068c10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efc28009910 tx=0x7efc280091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:56.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc39672700 1 -- 172.21.15.38:0/2772879088 shutdown_connections 2024-08-15T17:58:56.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc39672700 1 --2- 172.21.15.38:0/2772879088 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efc340af3f0 0x7efc34068c10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:56.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc39672700 1 -- 172.21.15.38:0/2772879088 >> 172.21.15.38:0/2772879088 conn(0x7efc3406d800 msgr2=0x7efc3406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:56.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc39672700 1 -- 172.21.15.38:0/2772879088 shutdown_connections 2024-08-15T17:58:56.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.649+0000 7efc39672700 1 -- 172.21.15.38:0/2772879088 wait complete. 2024-08-15T17:58:56.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc39672700 1 Processor -- start 2024-08-15T17:58:56.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc39672700 1 -- start start 2024-08-15T17:58:56.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc39672700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efc34068840 0x7efc34151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:56.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc39672700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc340af3f0 0x7efc34158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:56.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc39672700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc340b0a70 0x7efc34158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:56.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc39672700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc340cb020 con 0x7efc340b0a70 2024-08-15T17:58:56.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc39672700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc34152d90 con 0x7efc340af3f0 2024-08-15T17:58:56.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc39672700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc340c29d0 con 0x7efc34068840 2024-08-15T17:58:56.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc337fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc340af3f0 0x7efc34158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:56.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc38e71700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc340b0a70 0x7efc34158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:56.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc33fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efc34068840 0x7efc34151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:56.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc337fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc340af3f0 0x7efc34158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:40818/0 (socket says 172.21.15.38:40818) 2024-08-15T17:58:56.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc337fe700 1 -- 172.21.15.38:0/873752338 learned_addr learned my addr 172.21.15.38:0/873752338 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc38e71700 1 -- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efc34068840 msgr2=0x7efc34151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc38e71700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efc34068840 0x7efc34151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc38e71700 1 -- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc340af3f0 msgr2=0x7efc34158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc38e71700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc340af3f0 0x7efc34158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc38e71700 1 -- 172.21.15.38:0/873752338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc28009420 con 0x7efc340b0a70 2024-08-15T17:58:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc38e71700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc340b0a70 0x7efc34158850 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0011e0 tx=0x7efc2c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc1bfff700 1 -- 172.21.15.38:0/873752338 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c005b30 con 0x7efc340b0a70 2024-08-15T17:58:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc1bfff700 1 -- 172.21.15.38:0/873752338 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c005cb0 con 0x7efc340b0a70 2024-08-15T17:58:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc39672700 1 -- 172.21.15.38:0/873752338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc340c2cb0 con 0x7efc340b0a70 2024-08-15T17:58:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.653+0000 7efc1bfff700 1 -- 172.21.15.38:0/873752338 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c007320 con 0x7efc340b0a70 2024-08-15T17:58:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.657+0000 7efc39672700 1 -- 172.21.15.38:0/873752338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc341532c0 con 0x7efc340b0a70 2024-08-15T17:58:56.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.657+0000 7efc1bfff700 1 -- 172.21.15.38:0/873752338 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7efc2c007690 con 0x7efc340b0a70 2024-08-15T17:58:56.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.657+0000 7efc39672700 1 -- 172.21.15.38:0/873752338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc341534e0 con 0x7efc340b0a70 2024-08-15T17:58:56.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.657+0000 7efc1bfff700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efc20060860 0x7efc20062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:56.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.657+0000 7efc1bfff700 1 -- 172.21.15.38:0/873752338 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efc2c07bf50 con 0x7efc340b0a70 2024-08-15T17:58:56.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.657+0000 7efc33fff700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efc20060860 0x7efc20062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:56.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.661+0000 7efc33fff700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efc20060860 0x7efc20062d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc240040a0 tx=0x7efc24007f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.665+0000 7efc1bfff700 1 -- 172.21.15.38:0/873752338 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7efc2c04c300 con 0x7efc340b0a70 2024-08-15T17:58:56.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.865+0000 7efc39672700 1 -- 172.21.15.38:0/873752338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efc340638f0 con 0x7efc340b0a70 2024-08-15T17:58:56.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.865+0000 7efc1bfff700 1 -- 172.21.15.38:0/873752338 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7efc2c017060 con 0x7efc340b0a70 2024-08-15T17:58:56.870 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:58:56.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 -- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efc20060860 msgr2=0x7efc20062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:56.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efc20060860 0x7efc20062d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc240040a0 tx=0x7efc24007f90 comp rx=0 tx=0).stop 2024-08-15T17:58:56.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 -- 172.21.15.38:0/873752338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc340b0a70 msgr2=0x7efc34158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:56.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc340b0a70 0x7efc34158850 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0011e0 tx=0x7efc2c007d10 comp rx=0 tx=0).stop 2024-08-15T17:58:56.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 -- 172.21.15.38:0/873752338 shutdown_connections 2024-08-15T17:58:56.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efc20060860 0x7efc20062d10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:56.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc340b0a70 0x7efc34158850 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:56.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc340af3f0 0x7efc34158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:56.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 --2- 172.21.15.38:0/873752338 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efc34068840 0x7efc34151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:56.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 -- 172.21.15.38:0/873752338 >> 172.21.15.38:0/873752338 conn(0x7efc3406d800 msgr2=0x7efc34156340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:56.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 -- 172.21.15.38:0/873752338 shutdown_connections 2024-08-15T17:58:56.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:56.869+0000 7efc39672700 1 -- 172.21.15.38:0/873752338 wait complete. 2024-08-15T17:58:56.884 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:58:56.884 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:58:56.884 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-08-15T17:58:56.884 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-08-15T17:58:56.884 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-08-15T17:58:56.884 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:58:56.884 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:58:56.884 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:58:56.884 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-08-15T17:58:56.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:56.885 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:56.893 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42600sec preferred_lft 42600sec 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:51:73:f0:c7 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 2a:69:3f:88:03:6f brd ff:ff:ff:ff:ff:ff 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:56.894 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::7049:8aff:fe98:12d7/64 scope link 2024-08-15T17:58:56.895 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:56.895 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:58:56.895 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 2a:69:3f:88:03:6f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T17:58:56.895 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::2869:3fff:fe88:36f/64 scope link 2024-08-15T17:58:56.895 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:56.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:56.897 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:58:56.912 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-08-15T17:58:56.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:56.914 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-08-15T17:58:56.924 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-08-15T17:58:56.924 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:58:56.924 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:58:56.924 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:58:56.924 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:58:56.924 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:58:56.924 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:58:56.924 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:58:56.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:56.925 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2024-08-15T17:58:56.925 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-08-15T17:58:56.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:58:56.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:58:56.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:58:56.925 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-08-15T17:58:57.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:57.003 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-08-15T17:58:57.010 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-15T17:58:57.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:57.010 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list-id 2024-08-15T17:58:57.016 INFO:teuthology.orchestra.run.smithi104.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-08-15T17:58:57.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:57.017 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2024-08-15T17:58:57.017 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-08-15T17:58:57.017 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-15T17:58:57.017 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-08-15T17:58:57.017 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-08-15T17:58:57.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:57.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-08-15T17:58:57.102 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:58:57.102 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:58:57.103 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:58:57.103 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:57.103 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-08-15T17:58:57.103 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:57.103 INFO:teuthology.orchestra.run.smithi104.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:58:57.103 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 4a:c8:ee:b5:f6:8c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:58:57.103 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:58:57.103 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:57.103 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::48c8:eeff:feb5:f68c/64 scope link 2024-08-15T17:58:57.103 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:57.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:57.104 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-08-15T17:58:57.118 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:58:57.118 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:58:57.119 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:58:57.119 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:57.119 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-08-15T17:58:57.119 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:57.119 INFO:teuthology.orchestra.run.smithi104.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:58:57.119 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 4a:c8:ee:b5:f6:8c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:58:57.119 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:58:57.119 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:57.119 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::48c8:eeff:feb5:f68c/64 scope link 2024-08-15T17:58:57.119 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:58:57.120 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-08-15T17:58:57.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:57.120 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2024-08-15T17:58:57.120 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-08-15T17:58:57.120 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-08-15T17:58:57.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-08-15T17:58:57.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-08-15T17:58:57.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-08-15T17:58:57.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-08-15T17:58:57.121 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-08-15T17:58:57.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:58:57.178 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2024-08-15T17:58:57.178 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-08-15T17:58:57.178 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.1 up 2024-08-15T17:58:57.178 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set dev brx.1 master ceph-brx 2024-08-15T17:58:57.178 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-08-15T17:58:57.204 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-08-15T17:58:57.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:58:57.204 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-08-15T17:58:57.210 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-08-15T17:58:57.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:58:57.211 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-08-15T17:58:57.216 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2024-08-15T17:58:57.274 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:57.274 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:57.284 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:57.285 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:57.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:57.286 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2024-08-15T17:58:57.291 INFO:teuthology.orchestra.run.smithi104.stdout:52 2024-08-15T17:58:57.291 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-08-15T17:58:57.291 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-08-15T17:58:57.293 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2024-08-15T17:58:57.311 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:57.311 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:57.367 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:57.368 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:57.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:57.369 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2024-08-15T17:58:57.376 INFO:teuthology.orchestra.run.smithi104.stdout:52 2024-08-15T17:58:57.378 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi104.stderr:2024-08-15T17:58:57.377+0000 7fdfcb770380 -1 init, newargv = 0x55b4ec2543a0 newargc=15 2024-08-15T17:58:57.378 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi104.stderr:ceph-fuse[30055]: starting ceph client 2024-08-15T17:58:57.386 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi104.stderr:ceph-fuse[30055]: starting fuse 2024-08-15T17:58:58.478 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2024-08-15T17:58:58.494 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:58:58.494 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:58:58.551 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:58:58.552 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:58:58.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:58.553 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2024-08-15T17:58:58.558 INFO:teuthology.orchestra.run.smithi104.stdout:52 2024-08-15T17:58:58.558 INFO:teuthology.orchestra.run.smithi104.stdout:53 2024-08-15T17:58:58.558 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-08-15T17:58:58.558 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> import re 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> import os 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2024-08-15T17:58:58.559 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:58:58.560 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2024-08-15T17:58:58.560 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:58:58.560 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:58:58.560 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:58:58.560 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2024-08-15T17:58:58.560 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2024-08-15T17:58:58.560 DEBUG:teuthology.orchestra.run.smithi104:> return f 2024-08-15T17:58:58.560 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:58:58.560 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:58:58.560 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.3")) 2024-08-15T17:58:58.560 DEBUG:teuthology.orchestra.run.smithi104:> ' 2024-08-15T17:58:58.641 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.3.30055.asok 2024-08-15T17:58:58.824 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.30055.asok 2024-08-15T17:58:58.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:58.824 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.30055.asok status 2024-08-15T17:58:58.964 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-08-15T17:58:58.964 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2024-08-15T17:58:58.964 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:58:58.965 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:58:58.965 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "3", 2024-08-15T17:58:58.965 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2024-08-15T17:58:58.965 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-08-15T17:58:58.965 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "30055", 2024-08-15T17:58:58.965 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2024-08-15T17:58:58.965 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:58:58.965 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2024-08-15T17:58:58.966 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2024-08-15T17:58:58.966 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 4562, 2024-08-15T17:58:58.966 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2024-08-15T17:58:58.966 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2024-08-15T17:58:58.966 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2024-08-15T17:58:58.966 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 4562 2024-08-15T17:58:58.966 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:58:58.966 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2024-08-15T17:58:58.966 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2716741029 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2716741029 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.4562 192.168.0.2:0/2716741029", 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "192.168.0.2:0/2716741029", 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 7, 2024-08-15T17:58:58.967 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 21, 2024-08-15T17:58:58.968 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:58:58.968 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2024-08-15T17:58:58.968 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2024-08-15T17:58:58.968 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-08-15T17:58:58.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:58.973 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-08-15T17:58:58.973 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-08-15T17:58:58.983 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-08-15T17:58:58.984 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-08-15T17:58:58.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:58.984 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-15T17:58:59.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:59.004 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-08-15T17:58:59.004 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.2 2024-08-15T17:58:59.011 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-08-15T17:58:59.012 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-08-15T17:58:59.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:59.012 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-08-15T17:58:59.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:59.076 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2024-08-15T17:58:59.076 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.1 2024-08-15T17:58:59.084 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2024-08-15T17:58:59.084 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-08-15T17:58:59.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:59.085 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:58:59.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:59.141 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2024-08-15T17:58:59.141 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.3 2024-08-15T17:58:59.148 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2024-08-15T17:58:59.148 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-08-15T17:58:59.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:58:59.148 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-08-15T17:58:59.162 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-08-15T17:58:59.179 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:58:59.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.293+0000 7fefe72cb700 1 Processor -- start 2024-08-15T17:58:59.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.293+0000 7fefe72cb700 1 -- start start 2024-08-15T17:58:59.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe72cb700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe0142c30 0x7fefe0159be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe72cb700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe00b2220 con 0x7fefe006ee30 2024-08-15T17:58:59.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe72cb700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe00b2500 con 0x7fefe0142860 2024-08-15T17:58:59.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe72cb700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0159790 con 0x7fefe0142c30 2024-08-15T17:58:59.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe6aca700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fefe006ee30 0x7fefe015a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41012/0 (socket says 172.21.15.38:41012) 2024-08-15T17:58:59.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe6aca700 1 -- 172.21.15.38:0/183977193 learned_addr learned my addr 172.21.15.38:0/183977193 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:59.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe5ac8700 1 --2- 172.21.15.38:0/183977193 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe0142c30 0x7fefe0159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe52c7700 1 -- 172.21.15.38:0/183977193 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951223280 0 0) 0x7fefe00b2220 con 0x7fefe006ee30 2024-08-15T17:58:59.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe52c7700 1 -- 172.21.15.38:0/183977193 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefcc003570 con 0x7fefe006ee30 2024-08-15T17:58:59.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe52c7700 1 -- 172.21.15.38:0/183977193 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000189363 0 0) 0x7fefe00b2500 con 0x7fefe0142860 2024-08-15T17:58:59.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe52c7700 1 -- 172.21.15.38:0/183977193 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefe00b2220 con 0x7fefe0142860 2024-08-15T17:58:59.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe5ac8700 1 -- 172.21.15.38:0/183977193 >> v1:172.21.15.104:6789/0 conn(0x7fefe0142860 legacy=0x7fefe006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe5ac8700 1 -- 172.21.15.38:0/183977193 >> v1:172.21.15.38:6789/0 conn(0x7fefe006ee30 legacy=0x7fefe015a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe5ac8700 1 -- 172.21.15.38:0/183977193 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe00b27e0 con 0x7fefe0142c30 2024-08-15T17:58:59.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe5ac8700 1 --2- 172.21.15.38:0/183977193 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe0142c30 0x7fefe0159be0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009910 tx=0x7fefd00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48f8599c2dd84fb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:59.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe52c7700 1 -- 172.21.15.38:0/183977193 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0011630 con 0x7fefe0142c30 2024-08-15T17:58:59.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe52c7700 1 -- 172.21.15.38:0/183977193 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd0011790 con 0x7fefe0142c30 2024-08-15T17:58:59.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe52c7700 1 -- 172.21.15.38:0/183977193 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0011b30 con 0x7fefe0142c30 2024-08-15T17:58:59.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe72cb700 1 -- 172.21.15.38:0/183977193 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe0142c30 msgr2=0x7fefe0159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe72cb700 1 --2- 172.21.15.38:0/183977193 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe0142c30 0x7fefe0159be0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009910 tx=0x7fefd00091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe72cb700 1 -- 172.21.15.38:0/183977193 shutdown_connections 2024-08-15T17:58:59.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe72cb700 1 --2- 172.21.15.38:0/183977193 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe0142c30 0x7fefe0159be0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe72cb700 1 -- 172.21.15.38:0/183977193 >> 172.21.15.38:0/183977193 conn(0x7fefe00ca2b0 msgr2=0x7fefe006e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:59.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe72cb700 1 -- 172.21.15.38:0/183977193 shutdown_connections 2024-08-15T17:58:59.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.297+0000 7fefe72cb700 1 -- 172.21.15.38:0/183977193 wait complete. 2024-08-15T17:58:59.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe72cb700 1 Processor -- start 2024-08-15T17:58:59.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe72cb700 1 -- start start 2024-08-15T17:58:59.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe72cb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe006ee30 0x7fefe00c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe72cb700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe0142860 0x7fefe00c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe72cb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefe0142c30 0x7fefe00c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe72cb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe00c53f0 con 0x7fefe006ee30 2024-08-15T17:58:59.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe72cb700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe00c5640 con 0x7fefe0142c30 2024-08-15T17:58:59.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe72cb700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe00c5920 con 0x7fefe0142860 2024-08-15T17:58:59.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe62c9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe006ee30 0x7fefe00c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe62c9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe006ee30 0x7fefe00c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38602/0 (socket says 172.21.15.38:38602) 2024-08-15T17:58:59.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe62c9700 1 -- 172.21.15.38:0/3564116074 learned_addr learned my addr 172.21.15.38:0/3564116074 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:59.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe5ac8700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe0142860 0x7fefe00c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe6aca700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefe0142c30 0x7fefe00c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe62c9700 1 -- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe0142860 msgr2=0x7fefe00c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe62c9700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe0142860 0x7fefe00c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe62c9700 1 -- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefe0142c30 msgr2=0x7fefe00c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe62c9700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefe0142c30 0x7fefe00c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe62c9700 1 -- 172.21.15.38:0/3564116074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd0009420 con 0x7fefe006ee30 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe6aca700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefe0142c30 0x7fefe00c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe62c9700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe006ee30 0x7fefe00c7fb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00cb10 tx=0x7fefdc002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefd67fc700 1 -- 172.21.15.38:0/3564116074 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc00a7e0 con 0x7fefe006ee30 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefd67fc700 1 -- 172.21.15.38:0/3564116074 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefdc00a940 con 0x7fefe006ee30 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe72cb700 1 -- 172.21.15.38:0/3564116074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe00c5c00 con 0x7fefe006ee30 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefd67fc700 1 -- 172.21.15.38:0/3564116074 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc00ad00 con 0x7fefe006ee30 2024-08-15T17:58:59.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.301+0000 7fefe72cb700 1 -- 172.21.15.38:0/3564116074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe00c61c0 con 0x7fefe006ee30 2024-08-15T17:58:59.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.305+0000 7fefe72cb700 1 -- 172.21.15.38:0/3564116074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe00643a0 con 0x7fefe006ee30 2024-08-15T17:58:59.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.305+0000 7fefd67fc700 1 -- 172.21.15.38:0/3564116074 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fefdc012630 con 0x7fefe006ee30 2024-08-15T17:58:59.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.305+0000 7fefd67fc700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefc4060860 0x7fefc4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.305+0000 7fefd67fc700 1 -- 172.21.15.38:0/3564116074 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fefdc07cdd0 con 0x7fefe006ee30 2024-08-15T17:58:59.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.305+0000 7fefe5ac8700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefc4060860 0x7fefc4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.305+0000 7fefe5ac8700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefc4060860 0x7fefc4062d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009860 tx=0x7fefd0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:59.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.313+0000 7fefd67fc700 1 -- 172.21.15.38:0/3564116074 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fefdc04d220 con 0x7fefe006ee30 2024-08-15T17:58:59.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.485+0000 7fefe72cb700 1 -- 172.21.15.38:0/3564116074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefe00c8b10 con 0x7fefe006ee30 2024-08-15T17:58:59.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.485+0000 7fefd67fc700 1 -- 172.21.15.38:0/3564116074 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fefdc00ae60 con 0x7fefe006ee30 2024-08-15T17:58:59.489 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:58:59.489 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:58:59.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 -- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefc4060860 msgr2=0x7fefc4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefc4060860 0x7fefc4062d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009860 tx=0x7fefd0017000 comp rx=0 tx=0).stop 2024-08-15T17:58:59.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 -- 172.21.15.38:0/3564116074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe006ee30 msgr2=0x7fefe00c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe006ee30 0x7fefe00c7fb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00cb10 tx=0x7fefdc002820 comp rx=0 tx=0).stop 2024-08-15T17:58:59.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 -- 172.21.15.38:0/3564116074 shutdown_connections 2024-08-15T17:58:59.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefc4060860 0x7fefc4062d10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefe0142c30 0x7fefe00c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe0142860 0x7fefe00c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 --2- 172.21.15.38:0/3564116074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe006ee30 0x7fefe00c7fb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 -- 172.21.15.38:0/3564116074 >> 172.21.15.38:0/3564116074 conn(0x7fefe00ca2b0 msgr2=0x7fefe00c1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:59.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 -- 172.21.15.38:0/3564116074 shutdown_connections 2024-08-15T17:58:59.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.489+0000 7fefe72cb700 1 -- 172.21.15.38:0/3564116074 wait complete. 2024-08-15T17:58:59.504 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:58:34.079267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:58:34.621969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:58:59.504 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:58:59.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec5177700 1 Processor -- start 2024-08-15T17:58:59.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec5177700 1 -- start start 2024-08-15T17:58:59.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec5177700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbec0068db0 0x7fbec01570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec5177700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbec00ae2d0 0x7fbec00ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec5177700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec00b23d0 con 0x7fbec0156cf0 2024-08-15T17:58:59.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec5177700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec00cc020 con 0x7fbec00ae2d0 2024-08-15T17:58:59.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec5177700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec00b1850 con 0x7fbec0068db0 2024-08-15T17:58:59.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbebffff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbec0156cf0 0x7fbec00686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41014/0 (socket says 172.21.15.38:41014) 2024-08-15T17:58:59.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbebffff700 1 -- 172.21.15.38:0/1492005005 learned_addr learned my addr 172.21.15.38:0/1492005005 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:58:59.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbebf7fe700 1 --2- 172.21.15.38:0/1492005005 >> v2:172.21.15.104:3300/0 conn(0x7fbec0068db0 0x7fbec01570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec4976700 1 --2- 172.21.15.38:0/1492005005 >> v2:172.21.15.38:3300/0 conn(0x7fbec00ae2d0 0x7fbec00ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbebeffd700 1 -- 172.21.15.38:0/1492005005 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963626996 0 0) 0x7fbec00b23d0 con 0x7fbec0156cf0 2024-08-15T17:58:59.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbebeffd700 1 -- 172.21.15.38:0/1492005005 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbea8003570 con 0x7fbec0156cf0 2024-08-15T17:58:59.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec4976700 1 -- 172.21.15.38:0/1492005005 >> v2:172.21.15.104:3300/0 conn(0x7fbec0068db0 msgr2=0x7fbec01570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:58:59.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec4976700 1 --2- 172.21.15.38:0/1492005005 >> v2:172.21.15.104:3300/0 conn(0x7fbec0068db0 0x7fbec01570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec4976700 1 -- 172.21.15.38:0/1492005005 >> v1:172.21.15.38:6789/0 conn(0x7fbec0156cf0 legacy=0x7fbec00686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec4976700 1 -- 172.21.15.38:0/1492005005 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec00aecb0 con 0x7fbec00ae2d0 2024-08-15T17:58:59.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.613+0000 7fbec4976700 1 --2- 172.21.15.38:0/1492005005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec00ae2d0 0x7fbec00ae770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbec006d690 tx=0x7fbeb8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31456143bed49f7f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:59.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbebeffd700 1 -- 172.21.15.38:0/1492005005 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800e070 con 0x7fbec00ae2d0 2024-08-15T17:58:59.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbebeffd700 1 -- 172.21.15.38:0/1492005005 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb800bab0 con 0x7fbec00ae2d0 2024-08-15T17:58:59.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbebeffd700 1 -- 172.21.15.38:0/1492005005 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800be70 con 0x7fbec00ae2d0 2024-08-15T17:58:59.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbec5177700 1 -- 172.21.15.38:0/1492005005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec00ae2d0 msgr2=0x7fbec00ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbec5177700 1 --2- 172.21.15.38:0/1492005005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec00ae2d0 0x7fbec00ae770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbec006d690 tx=0x7fbeb8009780 comp rx=0 tx=0).stop 2024-08-15T17:58:59.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbec5177700 1 -- 172.21.15.38:0/1492005005 shutdown_connections 2024-08-15T17:58:59.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbec5177700 1 --2- 172.21.15.38:0/1492005005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec00ae2d0 0x7fbec00ae770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbec5177700 1 --2- 172.21.15.38:0/1492005005 >> v2:172.21.15.104:3300/0 conn(0x7fbec0068db0 0x7fbec01570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbec5177700 1 -- 172.21.15.38:0/1492005005 >> 172.21.15.38:0/1492005005 conn(0x7fbec006e350 msgr2=0x7fbec006b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:59.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbec5177700 1 -- 172.21.15.38:0/1492005005 shutdown_connections 2024-08-15T17:58:59.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbec5177700 1 -- 172.21.15.38:0/1492005005 wait complete. 2024-08-15T17:58:59.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbec5177700 1 Processor -- start 2024-08-15T17:58:59.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.617+0000 7fbec5177700 1 -- start start 2024-08-15T17:58:59.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbec5177700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbec0068db0 0x7fbec01552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbec5177700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec00ae2d0 0x7fbec0155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbec5177700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0156cf0 0x7fbec015c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbec5177700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec015da50 con 0x7fbec00ae2d0 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbec5177700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec0153110 con 0x7fbec0156cf0 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbec5177700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec015d770 con 0x7fbec0068db0 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbebf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec00ae2d0 0x7fbec0155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbebf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec00ae2d0 0x7fbec0155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38620/0 (socket says 172.21.15.38:38620) 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbebffff700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbec0068db0 0x7fbec01552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbebf7fe700 1 -- 172.21.15.38:0/4264892282 learned_addr learned my addr 172.21.15.38:0/4264892282 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbec4976700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0156cf0 0x7fbec015c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbebf7fe700 1 -- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbec0068db0 msgr2=0x7fbec01552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbebf7fe700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbec0068db0 0x7fbec01552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbebf7fe700 1 -- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0156cf0 msgr2=0x7fbec015c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbebf7fe700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0156cf0 0x7fbec015c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbebf7fe700 1 -- 172.21.15.38:0/4264892282 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb8009a20 con 0x7fbec00ae2d0 2024-08-15T17:58:59.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbec4976700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0156cf0 0x7fbec015c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:58:59.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbebf7fe700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec00ae2d0 0x7fbec0155820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400e930 tx=0x7fbeb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:59.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbea7fff700 1 -- 172.21.15.38:0/4264892282 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4013680 con 0x7fbec00ae2d0 2024-08-15T17:58:59.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbea7fff700 1 -- 172.21.15.38:0/4264892282 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb40137e0 con 0x7fbec00ae2d0 2024-08-15T17:58:59.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbec5177700 1 -- 172.21.15.38:0/4264892282 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec0152ac0 con 0x7fbec00ae2d0 2024-08-15T17:58:59.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbea7fff700 1 -- 172.21.15.38:0/4264892282 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4013b80 con 0x7fbec00ae2d0 2024-08-15T17:58:59.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbec5177700 1 -- 172.21.15.38:0/4264892282 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec015c7b0 con 0x7fbec00ae2d0 2024-08-15T17:58:59.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbec5177700 1 -- 172.21.15.38:0/4264892282 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec0152920 con 0x7fbec00ae2d0 2024-08-15T17:58:59.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.621+0000 7fbea7fff700 1 -- 172.21.15.38:0/4264892282 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbeb400c460 con 0x7fbec00ae2d0 2024-08-15T17:58:59.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.625+0000 7fbea7fff700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbeac060860 0x7fbeac062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.625+0000 7fbea7fff700 1 -- 172.21.15.38:0/4264892282 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbeb407cce0 con 0x7fbec00ae2d0 2024-08-15T17:58:59.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.625+0000 7fbebffff700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbeac060860 0x7fbeac062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.625+0000 7fbebffff700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbeac060860 0x7fbeac062d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0007fd0 tx=0x7fbeb0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:59.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.633+0000 7fbea7fff700 1 -- 172.21.15.38:0/4264892282 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbec0152920 con 0x7fbec00ae2d0 2024-08-15T17:58:59.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.797+0000 7fbec5177700 1 -- 172.21.15.38:0/4264892282 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbec0152920 con 0x7fbec00ae2d0 2024-08-15T17:58:59.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.797+0000 7fbea7fff700 1 -- 172.21.15.38:0/4264892282 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbec0152920 con 0x7fbec00ae2d0 2024-08-15T17:58:59.802 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T17:58:59.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 -- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbeac060860 msgr2=0x7fbeac062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbeac060860 0x7fbeac062d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0007fd0 tx=0x7fbeb0009040 comp rx=0 tx=0).stop 2024-08-15T17:58:59.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 -- 172.21.15.38:0/4264892282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec00ae2d0 msgr2=0x7fbec0155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec00ae2d0 0x7fbec0155820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400e930 tx=0x7fbeb40091a0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 -- 172.21.15.38:0/4264892282 shutdown_connections 2024-08-15T17:58:59.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fbeac060860 0x7fbeac062d10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0156cf0 0x7fbec015c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec00ae2d0 0x7fbec0155820 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 --2- 172.21.15.38:0/4264892282 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbec0068db0 0x7fbec01552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 -- 172.21.15.38:0/4264892282 >> 172.21.15.38:0/4264892282 conn(0x7fbec006e350 msgr2=0x7fbec006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:59.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 -- 172.21.15.38:0/4264892282 shutdown_connections 2024-08-15T17:58:59.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.801+0000 7fbec5177700 1 -- 172.21.15.38:0/4264892282 wait complete. 2024-08-15T17:58:59.816 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:58:59.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71bfff700 1 Processor -- start 2024-08-15T17:58:59.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71bfff700 1 -- start start 2024-08-15T17:58:59.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc71c0b2710 0x7fc71c0af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71bfff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71c0cea40 con 0x7fc71c160880 2024-08-15T17:58:59.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71bfff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71c0af730 con 0x7fc71c160c50 2024-08-15T17:58:59.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71bfff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71c0cde30 con 0x7fc71c0b2710 2024-08-15T17:58:59.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71a7fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc71c0b2710 0x7fc71c0af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71a7fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc71c0b2710 0x7fc71c0af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38622/0 (socket says 172.21.15.38:38622) 2024-08-15T17:58:59.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71a7fc700 1 -- 172.21.15.38:0/1708305035 learned_addr learned my addr 172.21.15.38:0/1708305035 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:59.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71affd700 1 --1- 172.21.15.38:0/1708305035 >> v1:172.21.15.38:6789/0 conn(0x7fc71c160880 0x7fc71c0b2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41030/0 (socket says 172.21.15.38:41030) 2024-08-15T17:58:59.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71a7fc700 1 -- 172.21.15.38:0/1708305035 >> v1:172.21.15.104:6789/0 conn(0x7fc71c160c50 legacy=0x7fc71c066e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71a7fc700 1 -- 172.21.15.38:0/1708305035 >> v1:172.21.15.38:6789/0 conn(0x7fc71c160880 legacy=0x7fc71c0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71a7fc700 1 -- 172.21.15.38:0/1708305035 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c0ce110 con 0x7fc71c0b2710 2024-08-15T17:58:59.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc71a7fc700 1 --2- 172.21.15.38:0/1708305035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc71c0b2710 0x7fc71c0af1f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc7100099b0 tx=0x7fc710009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b96a96a1257f7d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:59.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc719ffb700 1 -- 172.21.15.38:0/1708305035 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71000b860 con 0x7fc71c0b2710 2024-08-15T17:58:59.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc719ffb700 1 -- 172.21.15.38:0/1708305035 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71000b9c0 con 0x7fc71c0b2710 2024-08-15T17:58:59.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.929+0000 7fc719ffb700 1 -- 172.21.15.38:0/1708305035 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71000bd60 con 0x7fc71c0b2710 2024-08-15T17:58:59.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.933+0000 7fc71bfff700 1 -- 172.21.15.38:0/1708305035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc71c0b2710 msgr2=0x7fc71c0af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.933+0000 7fc71bfff700 1 --2- 172.21.15.38:0/1708305035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc71c0b2710 0x7fc71c0af1f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc7100099b0 tx=0x7fc710009250 comp rx=0 tx=0).stop 2024-08-15T17:58:59.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.933+0000 7fc71bfff700 1 -- 172.21.15.38:0/1708305035 shutdown_connections 2024-08-15T17:58:59.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.933+0000 7fc71bfff700 1 --2- 172.21.15.38:0/1708305035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc71c0b2710 0x7fc71c0af1f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.933+0000 7fc71bfff700 1 -- 172.21.15.38:0/1708305035 >> 172.21.15.38:0/1708305035 conn(0x7fc71c0d0b50 msgr2=0x7fc71c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:58:59.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.933+0000 7fc71bfff700 1 -- 172.21.15.38:0/1708305035 shutdown_connections 2024-08-15T17:58:59.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.933+0000 7fc71bfff700 1 -- 172.21.15.38:0/1708305035 wait complete. 2024-08-15T17:58:59.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.933+0000 7fc71bfff700 1 Processor -- start 2024-08-15T17:58:59.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71bfff700 1 -- start start 2024-08-15T17:58:59.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71bfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc71c0b2710 0x7fc71c0c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc71c160880 0x7fc71c0c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc71c160c50 0x7fc71c0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71bfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c0c9940 con 0x7fc71c160c50 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71bfff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c15de60 con 0x7fc71c160880 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71bfff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c15e140 con 0x7fc71c0b2710 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc71c160c50 0x7fc71c0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc71c160c50 0x7fc71c0c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38634/0 (socket says 172.21.15.38:38634) 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71b7fe700 1 -- 172.21.15.38:0/3485295520 learned_addr learned my addr 172.21.15.38:0/3485295520 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71affd700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc71c0b2710 0x7fc71c0c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71a7fc700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc71c160880 0x7fc71c0c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71b7fe700 1 -- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc71c0b2710 msgr2=0x7fc71c0c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71b7fe700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc71c0b2710 0x7fc71c0c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71b7fe700 1 -- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc71c160880 msgr2=0x7fc71c0c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:58:59.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71b7fe700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc71c160880 0x7fc71c0c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:58:59.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71b7fe700 1 -- 172.21.15.38:0/3485295520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7100094f0 con 0x7fc71c160c50 2024-08-15T17:58:59.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71b7fe700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc71c160c50 0x7fc71c0c9400 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c840 tx=0x7fc714001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:59.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc702ffd700 1 -- 172.21.15.38:0/3485295520 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714012630 con 0x7fc71c160c50 2024-08-15T17:58:59.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc702ffd700 1 -- 172.21.15.38:0/3485295520 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7140127b0 con 0x7fc71c160c50 2024-08-15T17:58:59.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71bfff700 1 -- 172.21.15.38:0/3485295520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71c15e420 con 0x7fc71c160c50 2024-08-15T17:58:59.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc702ffd700 1 -- 172.21.15.38:0/3485295520 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714012b70 con 0x7fc71c160c50 2024-08-15T17:58:59.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.937+0000 7fc71bfff700 1 -- 172.21.15.38:0/3485295520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71c15e9e0 con 0x7fc71c160c50 2024-08-15T17:58:59.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.941+0000 7fc702ffd700 1 -- 172.21.15.38:0/3485295520 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc71401a3a0 con 0x7fc71c160c50 2024-08-15T17:58:59.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.941+0000 7fc71bfff700 1 -- 172.21.15.38:0/3485295520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71c15ec00 con 0x7fc71c160c50 2024-08-15T17:58:59.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.941+0000 7fc702ffd700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc708060860 0x7fc708062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:58:59.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.941+0000 7fc702ffd700 1 -- 172.21.15.38:0/3485295520 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc71407cd20 con 0x7fc71c160c50 2024-08-15T17:58:59.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.941+0000 7fc71affd700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc708060860 0x7fc708062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:58:59.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.941+0000 7fc71affd700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc708060860 0x7fc708062d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc70c004fd0 tx=0x7fc70c007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:58:59.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:58:59.949+0000 7fc702ffd700 1 -- 172.21.15.38:0/3485295520 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc71404d170 con 0x7fc71c160c50 2024-08-15T17:59:00.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.117+0000 7fc71bfff700 1 -- 172.21.15.38:0/3485295520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc71c0638f0 con 0x7fc71c160c50 2024-08-15T17:59:00.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.117+0000 7fc702ffd700 1 -- 172.21.15.38:0/3485295520 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc714050790 con 0x7fc71c160c50 2024-08-15T17:59:00.120 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T17:59:00.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 -- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc708060860 msgr2=0x7fc708062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc708060860 0x7fc708062d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc70c004fd0 tx=0x7fc70c007eb0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 -- 172.21.15.38:0/3485295520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc71c160c50 msgr2=0x7fc71c0c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc71c160c50 0x7fc71c0c9400 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c840 tx=0x7fc714001670 comp rx=0 tx=0).stop 2024-08-15T17:59:00.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 -- 172.21.15.38:0/3485295520 shutdown_connections 2024-08-15T17:59:00.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc708060860 0x7fc708062d10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc71c160c50 0x7fc71c0c9400 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc71c160880 0x7fc71c0c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 --2- 172.21.15.38:0/3485295520 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc71c0b2710 0x7fc71c0c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 -- 172.21.15.38:0/3485295520 >> 172.21.15.38:0/3485295520 conn(0x7fc71c0d0b50 msgr2=0x7fc71c0b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:00.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 -- 172.21.15.38:0/3485295520 shutdown_connections 2024-08-15T17:59:00.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.121+0000 7fc71bfff700 1 -- 172.21.15.38:0/3485295520 wait complete. 2024-08-15T17:59:00.141 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:59:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.253+0000 7f8660886700 1 Processor -- start 2024-08-15T17:59:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.253+0000 7f8660886700 1 -- start start 2024-08-15T17:59:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.253+0000 7f8660886700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f865c0b0ea0 0x7f865c0b1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.253+0000 7f8660886700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f865c0ae8e0 0x7f865c0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.253+0000 7f8660886700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c068dd0 con 0x7f865c0b31f0 2024-08-15T17:59:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.253+0000 7f8660886700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c0b3d60 con 0x7f865c0ae8e0 2024-08-15T17:59:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.253+0000 7f8660886700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0aed30 con 0x7f865c0b0ea0 2024-08-15T17:59:00.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.253+0000 7f8653fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f865c0ae8e0 0x7f865c0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.253+0000 7f865b59e700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f865c0b31f0 0x7f865c069260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:55732/0 (socket says 172.21.15.38:55732) 2024-08-15T17:59:00.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.253+0000 7f865b59e700 1 -- 172.21.15.38:0/3815207550 learned_addr learned my addr 172.21.15.38:0/3815207550 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:00.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.253+0000 7f865ad9d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f865c0b0ea0 0x7f865c0b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8653fff700 1 -- 172.21.15.38:0/3815207550 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f865c0b0ea0 msgr2=0x7f865c0b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:00.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8653fff700 1 --2- 172.21.15.38:0/3815207550 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f865c0b0ea0 0x7f865c0b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8653fff700 1 -- 172.21.15.38:0/3815207550 >> v1:172.21.15.104:6789/0 conn(0x7f865c0b31f0 legacy=0x7f865c069260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8653fff700 1 -- 172.21.15.38:0/3815207550 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c0b3a80 con 0x7f865c0ae8e0 2024-08-15T17:59:00.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8653fff700 1 --2- 172.21.15.38:0/3815207550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f865c0ae8e0 0x7f865c0b2cb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8644009910 tx=0x7f86440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9dfe0dc8aca805 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:00.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f865a59c700 1 -- 172.21.15.38:0/3815207550 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864400b640 con 0x7f865c0ae8e0 2024-08-15T17:59:00.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f865a59c700 1 -- 172.21.15.38:0/3815207550 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864400b7a0 con 0x7f865c0ae8e0 2024-08-15T17:59:00.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f865a59c700 1 -- 172.21.15.38:0/3815207550 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864400bb60 con 0x7f865c0ae8e0 2024-08-15T17:59:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8660886700 1 -- 172.21.15.38:0/3815207550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f865c0ae8e0 msgr2=0x7f865c0b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8660886700 1 --2- 172.21.15.38:0/3815207550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f865c0ae8e0 0x7f865c0b2cb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8644009910 tx=0x7f86440091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8660886700 1 -- 172.21.15.38:0/3815207550 shutdown_connections 2024-08-15T17:59:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8660886700 1 --2- 172.21.15.38:0/3815207550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f865c0ae8e0 0x7f865c0b2cb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8660886700 1 --2- 172.21.15.38:0/3815207550 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f865c0b0ea0 0x7f865c0b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8660886700 1 -- 172.21.15.38:0/3815207550 >> 172.21.15.38:0/3815207550 conn(0x7f865c0ce920 msgr2=0x7f865c06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8660886700 1 -- 172.21.15.38:0/3815207550 shutdown_connections 2024-08-15T17:59:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8660886700 1 -- 172.21.15.38:0/3815207550 wait complete. 2024-08-15T17:59:00.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8660886700 1 Processor -- start 2024-08-15T17:59:00.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.257+0000 7f8660886700 1 -- start start 2024-08-15T17:59:00.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8660886700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f865c0ae8e0 0x7f865c1426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8660886700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f865c0b0ea0 0x7f865c13e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8660886700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c0b31f0 0x7f865c0cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8660886700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1404a0 con 0x7f865c0b31f0 2024-08-15T17:59:00.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8660886700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c13f5f0 con 0x7f865c0b0ea0 2024-08-15T17:59:00.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8660886700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c13f890 con 0x7f865c0ae8e0 2024-08-15T17:59:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f865b59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c0b31f0 0x7f865c0cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f865b59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c0b31f0 0x7f865c0cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38640/0 (socket says 172.21.15.38:38640) 2024-08-15T17:59:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f865b59e700 1 -- 172.21.15.38:0/1604202066 learned_addr learned my addr 172.21.15.38:0/1604202066 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f865ad9d700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f865c0ae8e0 0x7f865c1426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8653fff700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f865c0b0ea0 0x7f865c13e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f865b59e700 1 -- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f865c0ae8e0 msgr2=0x7f865c1426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f865b59e700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f865c0ae8e0 0x7f865c1426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f865b59e700 1 -- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f865c0b0ea0 msgr2=0x7f865c13e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f865b59e700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f865c0b0ea0 0x7f865c13e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f865b59e700 1 -- 172.21.15.38:0/1604202066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8644009420 con 0x7f865c0b31f0 2024-08-15T17:59:00.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f865b59e700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c0b31f0 0x7f865c0cc9e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f865400cbc0 tx=0x7f8654001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:00.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8652ffd700 1 -- 172.21.15.38:0/1604202066 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400a780 con 0x7f865c0b31f0 2024-08-15T17:59:00.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8652ffd700 1 -- 172.21.15.38:0/1604202066 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865400a900 con 0x7f865c0b31f0 2024-08-15T17:59:00.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8660886700 1 -- 172.21.15.38:0/1604202066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c13fb70 con 0x7f865c0b31f0 2024-08-15T17:59:00.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8652ffd700 1 -- 172.21.15.38:0/1604202066 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400acc0 con 0x7f865c0b31f0 2024-08-15T17:59:00.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8660886700 1 -- 172.21.15.38:0/1604202066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c140130 con 0x7f865c0b31f0 2024-08-15T17:59:00.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8660886700 1 -- 172.21.15.38:0/1604202066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c140610 con 0x7f865c0b31f0 2024-08-15T17:59:00.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.261+0000 7f8652ffd700 1 -- 172.21.15.38:0/1604202066 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8654007300 con 0x7f865c0b31f0 2024-08-15T17:59:00.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.265+0000 7f8652ffd700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f863c060860 0x7f863c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.265+0000 7f8652ffd700 1 -- 172.21.15.38:0/1604202066 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f865407cd20 con 0x7f865c0b31f0 2024-08-15T17:59:00.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.265+0000 7f865ad9d700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f863c060860 0x7f863c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.265+0000 7f865ad9d700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f863c060860 0x7f863c062d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f864c009db0 tx=0x7f864c0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:00.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.269+0000 7f8652ffd700 1 -- 172.21.15.38:0/1604202066 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f865404d1f0 con 0x7f865c0b31f0 2024-08-15T17:59:00.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.433+0000 7f8660886700 1 -- 172.21.15.38:0/1604202066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f865c064330 con 0x7f865c0b31f0 2024-08-15T17:59:00.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8652ffd700 1 -- 172.21.15.38:0/1604202066 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8654050810 con 0x7f865c0b31f0 2024-08-15T17:59:00.439 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-08-15T17:59:00.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 -- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f863c060860 msgr2=0x7f863c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f863c060860 0x7f863c062d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f864c009db0 tx=0x7f864c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 -- 172.21.15.38:0/1604202066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c0b31f0 msgr2=0x7f865c0cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c0b31f0 0x7f865c0cc9e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f865400cbc0 tx=0x7f8654001970 comp rx=0 tx=0).stop 2024-08-15T17:59:00.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 -- 172.21.15.38:0/1604202066 shutdown_connections 2024-08-15T17:59:00.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f863c060860 0x7f863c062d10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c0b31f0 0x7f865c0cc9e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f865c0b0ea0 0x7f865c13e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 --2- 172.21.15.38:0/1604202066 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f865c0ae8e0 0x7f865c1426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 -- 172.21.15.38:0/1604202066 >> 172.21.15.38:0/1604202066 conn(0x7f865c0ce920 msgr2=0x7f865c0b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:00.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 -- 172.21.15.38:0/1604202066 shutdown_connections 2024-08-15T17:59:00.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.437+0000 7f8660886700 1 -- 172.21.15.38:0/1604202066 wait complete. 2024-08-15T17:59:00.453 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:59:00.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.557+0000 7f8d9131a700 1 Processor -- start 2024-08-15T17:59:00.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.557+0000 7f8d9131a700 1 -- start start 2024-08-15T17:59:00.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.557+0000 7f8d9131a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8d8c15a200 0x7f8d8c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.557+0000 7f8d9131a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8d8c0c8720 0x7f8d8c0abe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.557+0000 7f8d9131a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8d8c0ac340 0x7f8d8c0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.557+0000 7f8d9131a700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8c0ca3b0 con 0x7f8d8c0c8720 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.557+0000 7f8d9131a700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8c0ca690 con 0x7f8d8c0ac340 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.557+0000 7f8d9131a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0ca970 con 0x7f8d8c15a200 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.557+0000 7f8d8b7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8d8c0c8720 0x7f8d8c0abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8b7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8d8c0c8720 0x7f8d8c0abe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38648/0 (socket says 172.21.15.38:38648) 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8b7fe700 1 -- 172.21.15.38:0/2327989447 learned_addr learned my addr 172.21.15.38:0/2327989447 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d90b19700 1 --2- 172.21.15.38:0/2327989447 >> v2:172.21.15.104:3300/0 conn(0x7f8d8c0ac340 0x7f8d8c0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8bfff700 1 --2- 172.21.15.38:0/2327989447 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8d8c15a200 0x7f8d8c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8b7fe700 1 -- 172.21.15.38:0/2327989447 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8d8c15a200 msgr2=0x7f8d8c0b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8b7fe700 1 --2- 172.21.15.38:0/2327989447 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8d8c15a200 0x7f8d8c0b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8b7fe700 1 -- 172.21.15.38:0/2327989447 >> v2:172.21.15.104:3300/0 conn(0x7f8d8c0ac340 msgr2=0x7f8d8c0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8b7fe700 1 --2- 172.21.15.38:0/2327989447 >> v2:172.21.15.104:3300/0 conn(0x7f8d8c0ac340 0x7f8d8c0cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8b7fe700 1 -- 172.21.15.38:0/2327989447 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c0cac50 con 0x7f8d8c0c8720 2024-08-15T17:59:00.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8b7fe700 1 --2- 172.21.15.38:0/2327989447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d8c0c8720 0x7f8d8c0abe00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c06d790 tx=0x7f8d80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cd0d99b93c2805 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:00.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8affd700 1 -- 172.21.15.38:0/2327989447 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80009e70 con 0x7f8d8c0c8720 2024-08-15T17:59:00.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8affd700 1 -- 172.21.15.38:0/2327989447 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8000b860 con 0x7f8d8c0c8720 2024-08-15T17:59:00.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d8affd700 1 -- 172.21.15.38:0/2327989447 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8000bc00 con 0x7f8d8c0c8720 2024-08-15T17:59:00.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d9131a700 1 -- 172.21.15.38:0/2327989447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d8c0c8720 msgr2=0x7f8d8c0abe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d9131a700 1 --2- 172.21.15.38:0/2327989447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d8c0c8720 0x7f8d8c0abe00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c06d790 tx=0x7f8d80009250 comp rx=0 tx=0).stop 2024-08-15T17:59:00.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d9131a700 1 -- 172.21.15.38:0/2327989447 shutdown_connections 2024-08-15T17:59:00.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d9131a700 1 --2- 172.21.15.38:0/2327989447 >> v2:172.21.15.104:3300/0 conn(0x7f8d8c0ac340 0x7f8d8c0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d9131a700 1 --2- 172.21.15.38:0/2327989447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d8c0c8720 0x7f8d8c0abe00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d9131a700 1 --2- 172.21.15.38:0/2327989447 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8d8c15a200 0x7f8d8c0b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d9131a700 1 -- 172.21.15.38:0/2327989447 >> 172.21.15.38:0/2327989447 conn(0x7f8d8c06d800 msgr2=0x7f8d8c06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:00.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d9131a700 1 -- 172.21.15.38:0/2327989447 shutdown_connections 2024-08-15T17:59:00.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.561+0000 7f8d9131a700 1 -- 172.21.15.38:0/2327989447 wait complete. 2024-08-15T17:59:00.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d9131a700 1 Processor -- start 2024-08-15T17:59:00.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d9131a700 1 -- start start 2024-08-15T17:59:00.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d9131a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d8c0ac340 0x7f8d8c145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d9131a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d8c0c8720 0x7f8d8c0adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d9131a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8d8c15a200 0x7f8d8c0af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d9131a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c12cae0 con 0x7f8d8c0c8720 2024-08-15T17:59:00.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d9131a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c146520 con 0x7f8d8c0ac340 2024-08-15T17:59:00.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d9131a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c151a30 con 0x7f8d8c15a200 2024-08-15T17:59:00.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d8b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d8c0c8720 0x7f8d8c0adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d8b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d8c0c8720 0x7f8d8c0adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38662/0 (socket says 172.21.15.38:38662) 2024-08-15T17:59:00.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d8b7fe700 1 -- 172.21.15.38:0/1933285616 learned_addr learned my addr 172.21.15.38:0/1933285616 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:00.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d8bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d8c0ac340 0x7f8d8c145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d90b19700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8d8c15a200 0x7f8d8c0af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d8b7fe700 1 -- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8d8c15a200 msgr2=0x7f8d8c0af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d8b7fe700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8d8c15a200 0x7f8d8c0af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d8b7fe700 1 -- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d8c0ac340 msgr2=0x7f8d8c145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d8b7fe700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d8c0ac340 0x7f8d8c145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d8b7fe700 1 -- 172.21.15.38:0/1933285616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d800094f0 con 0x7f8d8c0c8720 2024-08-15T17:59:00.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.565+0000 7f8d8b7fe700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d8c0c8720 0x7f8d8c0adcf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c069650 tx=0x7f8d80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:00.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.569+0000 7f8d73fff700 1 -- 172.21.15.38:0/1933285616 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8000bbe0 con 0x7f8d8c0c8720 2024-08-15T17:59:00.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.569+0000 7f8d73fff700 1 -- 172.21.15.38:0/1933285616 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d80000c90 con 0x7f8d8c0c8720 2024-08-15T17:59:00.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.569+0000 7f8d9131a700 1 -- 172.21.15.38:0/1933285616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c151d10 con 0x7f8d8c0c8720 2024-08-15T17:59:00.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.569+0000 7f8d73fff700 1 -- 172.21.15.38:0/1933285616 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80011650 con 0x7f8d8c0c8720 2024-08-15T17:59:00.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.569+0000 7f8d9131a700 1 -- 172.21.15.38:0/1933285616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c1522d0 con 0x7f8d8c0c8720 2024-08-15T17:59:00.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.569+0000 7f8d9131a700 1 -- 172.21.15.38:0/1933285616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c0638f0 con 0x7f8d8c0c8720 2024-08-15T17:59:00.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.569+0000 7f8d73fff700 1 -- 172.21.15.38:0/1933285616 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8d80011a80 con 0x7f8d8c0c8720 2024-08-15T17:59:00.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.569+0000 7f8d73fff700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8d78060860 0x7f8d78062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.569+0000 7f8d73fff700 1 -- 172.21.15.38:0/1933285616 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d8007e070 con 0x7f8d8c0c8720 2024-08-15T17:59:00.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.569+0000 7f8d8bfff700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8d78060860 0x7f8d78062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.573+0000 7f8d8bfff700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8d78060860 0x7f8d78062d10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c006fd0 tx=0x7f8d7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:00.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.581+0000 7f8d73fff700 1 -- 172.21.15.38:0/1933285616 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8d800ae480 con 0x7f8d8c0c8720 2024-08-15T17:59:00.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.745+0000 7f8d9131a700 1 -- 172.21.15.38:0/1933285616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d8c06e620 con 0x7f8d8c0c8720 2024-08-15T17:59:00.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d73fff700 1 -- 172.21.15.38:0/1933285616 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f8d8004e420 con 0x7f8d8c0c8720 2024-08-15T17:59:00.751 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:00.751 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:59:00.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 -- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8d78060860 msgr2=0x7f8d78062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8d78060860 0x7f8d78062d10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c006fd0 tx=0x7f8d7c008040 comp rx=0 tx=0).stop 2024-08-15T17:59:00.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 -- 172.21.15.38:0/1933285616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d8c0c8720 msgr2=0x7f8d8c0adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d8c0c8720 0x7f8d8c0adcf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c069650 tx=0x7f8d80017000 comp rx=0 tx=0).stop 2024-08-15T17:59:00.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 -- 172.21.15.38:0/1933285616 shutdown_connections 2024-08-15T17:59:00.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8d78060860 0x7f8d78062d10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8d8c15a200 0x7f8d8c0af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d8c0c8720 0x7f8d8c0adcf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 --2- 172.21.15.38:0/1933285616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d8c0ac340 0x7f8d8c145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 -- 172.21.15.38:0/1933285616 >> 172.21.15.38:0/1933285616 conn(0x7f8d8c06d800 msgr2=0x7f8d8c150290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:00.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 -- 172.21.15.38:0/1933285616 shutdown_connections 2024-08-15T17:59:00.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.749+0000 7f8d9131a700 1 -- 172.21.15.38:0/1933285616 wait complete. 2024-08-15T17:59:00.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:58:34.079267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:58:34.621969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:59:00.766 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:59:00.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.877+0000 7f05d740b700 1 Processor -- start 2024-08-15T17:59:00.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.877+0000 7f05d740b700 1 -- start start 2024-08-15T17:59:00.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f05d00cc840 0x7f05d00ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b5750 0x7f05d00b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05d00cd150 con 0x7f05d00b2210 2024-08-15T17:59:00.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d00af570 con 0x7f05d00cc840 2024-08-15T17:59:00.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d0159ce0 con 0x7f05d00b5750 2024-08-15T17:59:00.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d49a6700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f05d00b2210 0x7f05d00b5040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41046/0 (socket says 172.21.15.38:41046) 2024-08-15T17:59:00.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d49a6700 1 -- 172.21.15.38:0/3939616928 learned_addr learned my addr 172.21.15.38:0/3939616928 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:00.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d51a7700 1 --2- 172.21.15.38:0/3939616928 >> v2:172.21.15.104:3300/0 conn(0x7f05d00cc840 0x7f05d00ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d59a8700 1 --2- 172.21.15.38:0/3939616928 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b5750 0x7f05d00b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05c7fff700 1 -- 172.21.15.38:0/3939616928 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903059616 0 0) 0x7f05d00cd150 con 0x7f05d00b2210 2024-08-15T17:59:00.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05c7fff700 1 -- 172.21.15.38:0/3939616928 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05bc003570 con 0x7f05d00b2210 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05c7fff700 1 -- 172.21.15.38:0/3939616928 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1123721423 0 0) 0x7f05bc003570 con 0x7f05d00b2210 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d51a7700 1 -- 172.21.15.38:0/3939616928 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b5750 msgr2=0x7f05d00b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d51a7700 1 --2- 172.21.15.38:0/3939616928 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b5750 0x7f05d00b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d51a7700 1 -- 172.21.15.38:0/3939616928 >> v1:172.21.15.38:6789/0 conn(0x7f05d00b2210 legacy=0x7f05d00b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d51a7700 1 -- 172.21.15.38:0/3939616928 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d0159fc0 con 0x7f05d00cc840 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d51a7700 1 --2- 172.21.15.38:0/3939616928 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05d00cc840 0x7f05d00ccc10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f05c00098e0 tx=0x7f05c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=609dceaeae62b54a server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05c7fff700 1 -- 172.21.15.38:0/3939616928 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc001470 con 0x7f05d00cc840 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05c7fff700 1 -- 172.21.15.38:0/3939616928 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c0011420 con 0x7f05d00cc840 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05c7fff700 1 -- 172.21.15.38:0/3939616928 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c00117a0 con 0x7f05d00cc840 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 -- 172.21.15.38:0/3939616928 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05d00cc840 msgr2=0x7f05d00ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 --2- 172.21.15.38:0/3939616928 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05d00cc840 0x7f05d00ccc10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f05c00098e0 tx=0x7f05c00091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 -- 172.21.15.38:0/3939616928 shutdown_connections 2024-08-15T17:59:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 --2- 172.21.15.38:0/3939616928 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b5750 0x7f05d00b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 --2- 172.21.15.38:0/3939616928 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05d00cc840 0x7f05d00ccc10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 -- 172.21.15.38:0/3939616928 >> 172.21.15.38:0/3939616928 conn(0x7f05d00682c0 msgr2=0x7f05d00686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 -- 172.21.15.38:0/3939616928 shutdown_connections 2024-08-15T17:59:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.881+0000 7f05d740b700 1 -- 172.21.15.38:0/3939616928 wait complete. 2024-08-15T17:59:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d740b700 1 Processor -- start 2024-08-15T17:59:00.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d740b700 1 -- start start 2024-08-15T17:59:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d740b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b2210 0x7f05d014b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d740b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05d00b5750 0x7f05d015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d740b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d00cc840 0x7f05d00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d740b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d015c980 con 0x7f05d00cc840 2024-08-15T17:59:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d740b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d014bc50 con 0x7f05d00b5750 2024-08-15T17:59:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d740b700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d015bb00 con 0x7f05d00b2210 2024-08-15T17:59:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d59a8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d00cc840 0x7f05d00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d51a7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b2210 0x7f05d014b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d51a7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b2210 0x7f05d014b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:60312/0 (socket says 172.21.15.38:60312) 2024-08-15T17:59:00.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d59a8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d00cc840 0x7f05d00b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38670/0 (socket says 172.21.15.38:38670) 2024-08-15T17:59:00.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d51a7700 1 -- 172.21.15.38:0/2398227852 learned_addr learned my addr 172.21.15.38:0/2398227852 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:00.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d49a6700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05d00b5750 0x7f05d015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d51a7700 1 -- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05d00b5750 msgr2=0x7f05d015ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d51a7700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05d00b5750 0x7f05d015ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d51a7700 1 -- 172.21.15.38:0/2398227852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d00cc840 msgr2=0x7f05d00b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:00.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d51a7700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d00cc840 0x7f05d00b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:00.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d51a7700 1 -- 172.21.15.38:0/2398227852 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c0009420 con 0x7f05d00b2210 2024-08-15T17:59:00.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d59a8700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d00cc840 0x7f05d00b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:00.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05d51a7700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b2210 0x7f05d014b710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05d00884f0 tx=0x7f05c0016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:00.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.885+0000 7f05c57fa700 1 -- 172.21.15.38:0/2398227852 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c001caf0 con 0x7f05d00b2210 2024-08-15T17:59:00.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.889+0000 7f05c57fa700 1 -- 172.21.15.38:0/2398227852 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c001cc70 con 0x7f05d00b2210 2024-08-15T17:59:00.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.889+0000 7f05d740b700 1 -- 172.21.15.38:0/2398227852 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d015c050 con 0x7f05d00b2210 2024-08-15T17:59:00.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.889+0000 7f05c57fa700 1 -- 172.21.15.38:0/2398227852 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c001a450 con 0x7f05d00b2210 2024-08-15T17:59:00.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.889+0000 7f05d740b700 1 -- 172.21.15.38:0/2398227852 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d015c610 con 0x7f05d00b2210 2024-08-15T17:59:00.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.889+0000 7f05d740b700 1 -- 172.21.15.38:0/2398227852 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d00645c0 con 0x7f05d00b2210 2024-08-15T17:59:00.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.889+0000 7f05c57fa700 1 -- 172.21.15.38:0/2398227852 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f05c00439d0 con 0x7f05d00b2210 2024-08-15T17:59:00.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.889+0000 7f05c57fa700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f05b4060890 0x7f05b4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:00.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.889+0000 7f05c57fa700 1 -- 172.21.15.38:0/2398227852 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f05c007e7c0 con 0x7f05d00b2210 2024-08-15T17:59:00.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.889+0000 7f05d49a6700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f05b4060890 0x7f05b4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:00.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.893+0000 7f05d49a6700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f05b4060890 0x7f05b4062d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f05d00aef50 tx=0x7f05cc001970 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:00.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:00.901+0000 7f05c57fa700 1 -- 172.21.15.38:0/2398227852 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f05c004ec90 con 0x7f05d00b2210 2024-08-15T17:59:01.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.065+0000 7f05d740b700 1 -- 172.21.15.38:0/2398227852 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05d00b1570 con 0x7f05d00b2210 2024-08-15T17:59:01.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05c57fa700 1 -- 172.21.15.38:0/2398227852 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05c00522b0 con 0x7f05d00b2210 2024-08-15T17:59:01.071 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T17:59:01.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 -- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f05b4060890 msgr2=0x7f05b4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f05b4060890 0x7f05b4062d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f05d00aef50 tx=0x7f05cc001970 comp rx=0 tx=0).stop 2024-08-15T17:59:01.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 -- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b2210 msgr2=0x7f05d014b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b2210 0x7f05d014b710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05d00884f0 tx=0x7f05c0016000 comp rx=0 tx=0).stop 2024-08-15T17:59:01.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 -- 172.21.15.38:0/2398227852 shutdown_connections 2024-08-15T17:59:01.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f05b4060890 0x7f05b4062d40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d00cc840 0x7f05d00b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05d00b5750 0x7f05d015ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 --2- 172.21.15.38:0/2398227852 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f05d00b2210 0x7f05d014b710 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 -- 172.21.15.38:0/2398227852 >> 172.21.15.38:0/2398227852 conn(0x7f05d00682c0 msgr2=0x7f05d00cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:01.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 -- 172.21.15.38:0/2398227852 shutdown_connections 2024-08-15T17:59:01.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.069+0000 7f05d740b700 1 -- 172.21.15.38:0/2398227852 wait complete. 2024-08-15T17:59:01.085 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:59:01.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8f359e700 1 Processor -- start 2024-08-15T17:59:01.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8f359e700 1 -- start start 2024-08-15T17:59:01.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8f359e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc8f4156e50 0x7fc8f40aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8f359e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8f40ae9e0 0x7fc8f40b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8f359e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8f40b22f0 0x7fc8f40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8f359e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f40afa80 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8f359e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f40aee70 con 0x7fc8f4156e50 2024-08-15T17:59:01.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8f359e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40af150 con 0x7fc8f40b22f0 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8ebfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8f40ae9e0 0x7fc8f40b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8ebfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8f40ae9e0 0x7fc8f40b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38676/0 (socket says 172.21.15.38:38676) 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8ebfff700 1 -- 172.21.15.38:0/1338436897 learned_addr learned my addr 172.21.15.38:0/1338436897 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8f259c700 1 --2- 172.21.15.38:0/1338436897 >> v2:172.21.15.104:3300/0 conn(0x7fc8f4156e50 0x7fc8f40aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.193+0000 7fc8f2d9d700 1 --2- 172.21.15.38:0/1338436897 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8f40b22f0 0x7fc8f40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8ebfff700 1 -- 172.21.15.38:0/1338436897 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8f40b22f0 msgr2=0x7fc8f40b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8ebfff700 1 --2- 172.21.15.38:0/1338436897 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8f40b22f0 0x7fc8f40b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8ebfff700 1 -- 172.21.15.38:0/1338436897 >> v2:172.21.15.104:3300/0 conn(0x7fc8f4156e50 msgr2=0x7fc8f40aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8ebfff700 1 --2- 172.21.15.38:0/1338436897 >> v2:172.21.15.104:3300/0 conn(0x7fc8f4156e50 0x7fc8f40aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8ebfff700 1 -- 172.21.15.38:0/1338436897 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f40af430 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f259c700 1 --2- 172.21.15.38:0/1338436897 >> v2:172.21.15.104:3300/0 conn(0x7fc8f4156e50 0x7fc8f40aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8ebfff700 1 --2- 172.21.15.38:0/1338436897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8f40ae9e0 0x7fc8f40b1db0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc8f406ed10 tx=0x7fc8dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bc1c76ccf391de server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f1d9b700 1 -- 172.21.15.38:0/1338436897 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc00b640 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f1d9b700 1 -- 172.21.15.38:0/1338436897 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc00b7a0 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f1d9b700 1 -- 172.21.15.38:0/1338436897 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc00bb60 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f359e700 1 -- 172.21.15.38:0/1338436897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8f40ae9e0 msgr2=0x7fc8f40b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f359e700 1 --2- 172.21.15.38:0/1338436897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8f40ae9e0 0x7fc8f40b1db0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc8f406ed10 tx=0x7fc8dc009250 comp rx=0 tx=0).stop 2024-08-15T17:59:01.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f359e700 1 -- 172.21.15.38:0/1338436897 shutdown_connections 2024-08-15T17:59:01.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f359e700 1 --2- 172.21.15.38:0/1338436897 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8f40b22f0 0x7fc8f40b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f359e700 1 --2- 172.21.15.38:0/1338436897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8f40ae9e0 0x7fc8f40b1db0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f359e700 1 --2- 172.21.15.38:0/1338436897 >> v2:172.21.15.104:3300/0 conn(0x7fc8f4156e50 0x7fc8f40aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f359e700 1 -- 172.21.15.38:0/1338436897 >> 172.21.15.38:0/1338436897 conn(0x7fc8f406cbe0 msgr2=0x7fc8f406cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:01.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f359e700 1 -- 172.21.15.38:0/1338436897 shutdown_connections 2024-08-15T17:59:01.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.197+0000 7fc8f359e700 1 -- 172.21.15.38:0/1338436897 wait complete. 2024-08-15T17:59:01.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8f359e700 1 Processor -- start 2024-08-15T17:59:01.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8f359e700 1 -- start start 2024-08-15T17:59:01.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8f359e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8f40ae9e0 0x7fc8f413cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8f359e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8f40b22f0 0x7fc8f40c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8f359e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8f4156e50 0x7fc8f416a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8f359e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f4158980 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8f359e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f4157900 con 0x7fc8f40b22f0 2024-08-15T17:59:01.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8f359e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f4157be0 con 0x7fc8f4156e50 2024-08-15T17:59:01.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8f259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8f40ae9e0 0x7fc8f413cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8f259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8f40ae9e0 0x7fc8f413cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38678/0 (socket says 172.21.15.38:38678) 2024-08-15T17:59:01.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8f259c700 1 -- 172.21.15.38:0/2709638114 learned_addr learned my addr 172.21.15.38:0/2709638114 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:01.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.201+0000 7fc8ebfff700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8f40b22f0 0x7fc8f40c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8f2d9d700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8f4156e50 0x7fc8f416a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8f259c700 1 -- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8f4156e50 msgr2=0x7fc8f416a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8f259c700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8f4156e50 0x7fc8f416a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8f259c700 1 -- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8f40b22f0 msgr2=0x7fc8f40c2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8f259c700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8f40b22f0 0x7fc8f40c2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8f259c700 1 -- 172.21.15.38:0/2709638114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e4009420 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8f2d9d700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8f4156e50 0x7fc8f416a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8f259c700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8f40ae9e0 0x7fc8f413cc90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400ec80 tx=0x7fc8e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8ea7fc700 1 -- 172.21.15.38:0/2709638114 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400c850 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8ea7fc700 1 -- 172.21.15.38:0/2709638114 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e400c9b0 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8f359e700 1 -- 172.21.15.38:0/2709638114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8dc0094f0 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8ea7fc700 1 -- 172.21.15.38:0/2709638114 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400cd30 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8f359e700 1 -- 172.21.15.38:0/2709638114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f4158330 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8f359e700 1 -- 172.21.15.38:0/2709638114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f40b35a0 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.205+0000 7fc8ea7fc700 1 -- 172.21.15.38:0/2709638114 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc8e4013680 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.209+0000 7fc8ea7fc700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc8d4060890 0x7fc8d4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.209+0000 7fc8ea7fc700 1 -- 172.21.15.38:0/2709638114 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc8e407d260 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.209+0000 7fc8ebfff700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc8d4060890 0x7fc8d4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.209+0000 7fc8ebfff700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc8d4060890 0x7fc8d4062d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc8f406ed10 tx=0x7fc8dc017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:01.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.217+0000 7fc8ea7fc700 1 -- 172.21.15.38:0/2709638114 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc8f40b35a0 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.377+0000 7fc8f359e700 1 -- 172.21.15.38:0/2709638114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8f40b35a0 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.377+0000 7fc8ea7fc700 1 -- 172.21.15.38:0/2709638114 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8f40b35a0 con 0x7fc8f40ae9e0 2024-08-15T17:59:01.382 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T17:59:01.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 -- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc8d4060890 msgr2=0x7fc8d4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc8d4060890 0x7fc8d4062d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc8f406ed10 tx=0x7fc8dc017000 comp rx=0 tx=0).stop 2024-08-15T17:59:01.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 -- 172.21.15.38:0/2709638114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8f40ae9e0 msgr2=0x7fc8f413cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8f40ae9e0 0x7fc8f413cc90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400ec80 tx=0x7fc8e40091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 -- 172.21.15.38:0/2709638114 shutdown_connections 2024-08-15T17:59:01.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc8d4060890 0x7fc8d4062d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8f4156e50 0x7fc8f416a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8f40b22f0 0x7fc8f40c2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 --2- 172.21.15.38:0/2709638114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8f40ae9e0 0x7fc8f413cc90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 -- 172.21.15.38:0/2709638114 >> 172.21.15.38:0/2709638114 conn(0x7fc8f406cbe0 msgr2=0x7fc8f40b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:01.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 -- 172.21.15.38:0/2709638114 shutdown_connections 2024-08-15T17:59:01.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.381+0000 7fc8f359e700 1 -- 172.21.15.38:0/2709638114 wait complete. 2024-08-15T17:59:01.396 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:59:01.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.513+0000 7fb449bda700 1 Processor -- start 2024-08-15T17:59:01.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.513+0000 7fb449bda700 1 -- start start 2024-08-15T17:59:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.513+0000 7fb449bda700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb444067510 0x7fb444164330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.513+0000 7fb449bda700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4440cba90 con 0x7fb4441596d0 2024-08-15T17:59:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.513+0000 7fb449bda700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4440cb160 con 0x7fb444159300 2024-08-15T17:59:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.513+0000 7fb449bda700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4440b2b60 con 0x7fb444067510 2024-08-15T17:59:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.513+0000 7fb442ffd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb4441596d0 0x7fb444066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41058/0 (socket says 172.21.15.38:41058) 2024-08-15T17:59:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.513+0000 7fb442ffd700 1 -- 172.21.15.38:0/2761691686 learned_addr learned my addr 172.21.15.38:0/2761691686 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:01.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.513+0000 7fb443fff700 1 --2- 172.21.15.38:0/2761691686 >> v2:172.21.15.104:3300/0 conn(0x7fb444067510 0x7fb444164330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.513+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793523039 0 0) 0x7fb4440cba90 con 0x7fb4441596d0 2024-08-15T17:59:01.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.513+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb42c003570 con 0x7fb4441596d0 2024-08-15T17:59:01.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317423884 0 0) 0x7fb4440cb160 con 0x7fb444159300 2024-08-15T17:59:01.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4440cba90 con 0x7fb444159300 2024-08-15T17:59:01.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694970326 0 0) 0x7fb42c003570 con 0x7fb4441596d0 2024-08-15T17:59:01.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 >> v2:172.21.15.104:3300/0 conn(0x7fb444067510 msgr2=0x7fb444164330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 --2- 172.21.15.38:0/2761691686 >> v2:172.21.15.104:3300/0 conn(0x7fb444067510 0x7fb444164330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 >> v1:172.21.15.104:6789/0 conn(0x7fb444159300 legacy=0x7fb4440b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4440cb440 con 0x7fb4441596d0 2024-08-15T17:59:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7fb434001470 con 0x7fb4441596d0 2024-08-15T17:59:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 >> v1:172.21.15.38:6789/0 conn(0x7fb4441596d0 legacy=0x7fb444066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 --2- 172.21.15.38:0/2761691686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb42c003bf0 0x7fb42c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 --2- 172.21.15.38:0/2761691686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb42c004810 0x7fb42c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 --2- 172.21.15.38:0/2761691686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb42c0051c0 0x7fb42c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440b2b60 con 0x7fb42c004810 2024-08-15T17:59:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c00c160 con 0x7fb42c003bf0 2024-08-15T17:59:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c00c320 con 0x7fb42c0051c0 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb442ffd700 1 --2- 172.21.15.38:0/2761691686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb42c004810 0x7fb42c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4437fe700 1 --2- 172.21.15.38:0/2761691686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb42c003bf0 0x7fb42c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb443fff700 1 --2- 172.21.15.38:0/2761691686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb42c0051c0 0x7fb42c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb442ffd700 1 -- 172.21.15.38:0/2761691686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb42c0051c0 msgr2=0x7fb42c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb442ffd700 1 --2- 172.21.15.38:0/2761691686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb42c0051c0 0x7fb42c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb442ffd700 1 -- 172.21.15.38:0/2761691686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb42c003bf0 msgr2=0x7fb42c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb442ffd700 1 --2- 172.21.15.38:0/2761691686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb42c003bf0 0x7fb42c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb443fff700 1 -- 172.21.15.38:0/2761691686 reap_dead start 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb442ffd700 1 -- 172.21.15.38:0/2761691686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c003570 con 0x7fb42c004810 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb442ffd700 1 --2- 172.21.15.38:0/2761691686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb42c004810 0x7fb42c004c80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb43400c2a0 tx=0x7fb434001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a151e1f2084cd89e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43400cb50 con 0x7fb42c004810 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43400ccb0 con 0x7fb42c004810 2024-08-15T17:59:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb4427fc700 1 -- 172.21.15.38:0/2761691686 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb434005b70 con 0x7fb42c004810 2024-08-15T17:59:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb449bda700 1 -- 172.21.15.38:0/2761691686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb42c004810 msgr2=0x7fb42c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb449bda700 1 --2- 172.21.15.38:0/2761691686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb42c004810 0x7fb42c004c80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb43400c2a0 tx=0x7fb434001e10 comp rx=0 tx=0).stop 2024-08-15T17:59:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb449bda700 1 -- 172.21.15.38:0/2761691686 shutdown_connections 2024-08-15T17:59:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb449bda700 1 --2- 172.21.15.38:0/2761691686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb42c004810 0x7fb42c004c80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb449bda700 1 -- 172.21.15.38:0/2761691686 >> 172.21.15.38:0/2761691686 conn(0x7fb4440682c0 msgr2=0x7fb4440686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb449bda700 1 -- 172.21.15.38:0/2761691686 shutdown_connections 2024-08-15T17:59:01.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.517+0000 7fb449bda700 1 -- 172.21.15.38:0/2761691686 wait complete. 2024-08-15T17:59:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb449bda700 1 Processor -- start 2024-08-15T17:59:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb449bda700 1 -- start start 2024-08-15T17:59:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb449bda700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb42c004810 0x7fb444155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb449bda700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4440b0340 0x7fb4440c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb449bda700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4440c6030 0x7fb4440cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb449bda700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440c4730 con 0x7fb4440c6030 2024-08-15T17:59:01.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb449bda700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb444155f90 con 0x7fb4440b0340 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb449bda700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44415aa50 con 0x7fb42c004810 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb443fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4440c6030 0x7fb4440cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb443fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4440c6030 0x7fb4440cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38696/0 (socket says 172.21.15.38:38696) 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb443fff700 1 -- 172.21.15.38:0/292295439 learned_addr learned my addr 172.21.15.38:0/292295439 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb442ffd700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4440b0340 0x7fb4440c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb4437fe700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb42c004810 0x7fb444155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb443fff700 1 -- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb42c004810 msgr2=0x7fb444155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb443fff700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb42c004810 0x7fb444155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb443fff700 1 -- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4440b0340 msgr2=0x7fb4440c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb443fff700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4440b0340 0x7fb4440c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb443fff700 1 -- 172.21.15.38:0/292295439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4440c6d90 con 0x7fb4440c6030 2024-08-15T17:59:01.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb442ffd700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4440b0340 0x7fb4440c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:01.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb443fff700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4440c6030 0x7fb4440cf1e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb444067910 tx=0x7fb43c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:01.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb42b7fe700 1 -- 172.21.15.38:0/292295439 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c00e070 con 0x7fb4440c6030 2024-08-15T17:59:01.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb42b7fe700 1 -- 172.21.15.38:0/292295439 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c011c30 con 0x7fb4440c6030 2024-08-15T17:59:01.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb449bda700 1 -- 172.21.15.38:0/292295439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4440c64f0 con 0x7fb4440c6030 2024-08-15T17:59:01.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb42b7fe700 1 -- 172.21.15.38:0/292295439 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c016470 con 0x7fb4440c6030 2024-08-15T17:59:01.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.521+0000 7fb449bda700 1 -- 172.21.15.38:0/292295439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4440c6ab0 con 0x7fb4440c6030 2024-08-15T17:59:01.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.525+0000 7fb449bda700 1 -- 172.21.15.38:0/292295439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4440645c0 con 0x7fb4440c6030 2024-08-15T17:59:01.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.525+0000 7fb42b7fe700 1 -- 172.21.15.38:0/292295439 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb43c0168a0 con 0x7fb4440c6030 2024-08-15T17:59:01.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.525+0000 7fb42b7fe700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb430060860 0x7fb430062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.525+0000 7fb42b7fe700 1 -- 172.21.15.38:0/292295439 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb43c07d730 con 0x7fb4440c6030 2024-08-15T17:59:01.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.525+0000 7fb4437fe700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb430060860 0x7fb430062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.525+0000 7fb4437fe700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb430060860 0x7fb430062d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb438000c00 tx=0x7fb438007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:01.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.533+0000 7fb42b7fe700 1 -- 172.21.15.38:0/292295439 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb43c04dc40 con 0x7fb4440c6030 2024-08-15T17:59:01.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.697+0000 7fb449bda700 1 -- 172.21.15.38:0/292295439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb44415ab90 con 0x7fb4440c6030 2024-08-15T17:59:01.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.697+0000 7fb42b7fe700 1 -- 172.21.15.38:0/292295439 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb43c009780 con 0x7fb4440c6030 2024-08-15T17:59:01.700 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-08-15T17:59:01.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 -- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb430060860 msgr2=0x7fb430062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb430060860 0x7fb430062d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb438000c00 tx=0x7fb438007d30 comp rx=0 tx=0).stop 2024-08-15T17:59:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 -- 172.21.15.38:0/292295439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4440c6030 msgr2=0x7fb4440cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4440c6030 0x7fb4440cf1e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb444067910 tx=0x7fb43c00b040 comp rx=0 tx=0).stop 2024-08-15T17:59:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 -- 172.21.15.38:0/292295439 shutdown_connections 2024-08-15T17:59:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb430060860 0x7fb430062d10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4440c6030 0x7fb4440cf1e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4440b0340 0x7fb4440c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 --2- 172.21.15.38:0/292295439 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb42c004810 0x7fb444155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 -- 172.21.15.38:0/292295439 >> 172.21.15.38:0/292295439 conn(0x7fb4440682c0 msgr2=0x7fb4441567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 -- 172.21.15.38:0/292295439 shutdown_connections 2024-08-15T17:59:01.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.701+0000 7fb449bda700 1 -- 172.21.15.38:0/292295439 wait complete. 2024-08-15T17:59:01.721 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:59:01.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.829+0000 7f0672102700 1 Processor -- start 2024-08-15T17:59:01.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.829+0000 7f0672102700 1 -- start start 2024-08-15T17:59:01.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.829+0000 7f0672102700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f066c158eb0 0x7f066c0aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.829+0000 7f0672102700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f066c06e620 0x7f066c0ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.829+0000 7f0672102700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066c0ac000 con 0x7f066c0aecc0 2024-08-15T17:59:01.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.829+0000 7f0672102700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066c0ab3f0 con 0x7f066c06e620 2024-08-15T17:59:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.829+0000 7f0672102700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066c0ab6d0 con 0x7f066c158eb0 2024-08-15T17:59:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.829+0000 7f06708ff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f066c0aecc0 0x7f066c06df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41062/0 (socket says 172.21.15.38:41062) 2024-08-15T17:59:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.829+0000 7f06708ff700 1 -- 172.21.15.38:0/2980387275 learned_addr learned my addr 172.21.15.38:0/2980387275 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.829+0000 7f0671901700 1 --2- 172.21.15.38:0/2980387275 >> v2:172.21.15.38:3300/0 conn(0x7f066c06e620 0x7f066c0ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.829+0000 7f0671100700 1 --2- 172.21.15.38:0/2980387275 >> v2:172.21.15.104:3300/0 conn(0x7f066c158eb0 0x7f066c0aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0671901700 1 -- 172.21.15.38:0/2980387275 >> v2:172.21.15.104:3300/0 conn(0x7f066c158eb0 msgr2=0x7f066c0aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0671901700 1 --2- 172.21.15.38:0/2980387275 >> v2:172.21.15.104:3300/0 conn(0x7f066c158eb0 0x7f066c0aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0671901700 1 -- 172.21.15.38:0/2980387275 >> v1:172.21.15.38:6789/0 conn(0x7f066c0aecc0 legacy=0x7f066c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0671901700 1 -- 172.21.15.38:0/2980387275 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0ab9b0 con 0x7f066c06e620 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0671901700 1 --2- 172.21.15.38:0/2980387275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c06e620 0x7f066c0ac490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f066c06c630 tx=0x7f0664009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63480a58904a86d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0663fff700 1 -- 172.21.15.38:0/2980387275 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066400e070 con 0x7f066c06e620 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0663fff700 1 -- 172.21.15.38:0/2980387275 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066400ba60 con 0x7f066c06e620 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0663fff700 1 -- 172.21.15.38:0/2980387275 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066400be20 con 0x7f066c06e620 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0672102700 1 -- 172.21.15.38:0/2980387275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c06e620 msgr2=0x7f066c0ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0672102700 1 --2- 172.21.15.38:0/2980387275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c06e620 0x7f066c0ac490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f066c06c630 tx=0x7f0664009780 comp rx=0 tx=0).stop 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0672102700 1 -- 172.21.15.38:0/2980387275 shutdown_connections 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0672102700 1 --2- 172.21.15.38:0/2980387275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c06e620 0x7f066c0ac490 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0672102700 1 --2- 172.21.15.38:0/2980387275 >> v2:172.21.15.104:3300/0 conn(0x7f066c158eb0 0x7f066c0aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0672102700 1 -- 172.21.15.38:0/2980387275 >> 172.21.15.38:0/2980387275 conn(0x7f066c0678d0 msgr2=0x7f066c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:01.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0672102700 1 -- 172.21.15.38:0/2980387275 shutdown_connections 2024-08-15T17:59:01.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0672102700 1 -- 172.21.15.38:0/2980387275 wait complete. 2024-08-15T17:59:01.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0672102700 1 Processor -- start 2024-08-15T17:59:01.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.833+0000 7f0672102700 1 -- start start 2024-08-15T17:59:01.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0672102700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f066c06e620 0x7f066c14a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0672102700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c0aecc0 0x7f066c0b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0672102700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f066c158eb0 0x7f066c0b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0672102700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c155310 con 0x7f066c0aecc0 2024-08-15T17:59:01.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0672102700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c16cb60 con 0x7f066c06e620 2024-08-15T17:59:01.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0672102700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c16d6e0 con 0x7f066c158eb0 2024-08-15T17:59:01.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06708ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c0aecc0 0x7f066c0b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06708ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c0aecc0 0x7f066c0b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38710/0 (socket says 172.21.15.38:38710) 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06708ff700 1 -- 172.21.15.38:0/2367959243 learned_addr learned my addr 172.21.15.38:0/2367959243 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0671901700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f066c158eb0 0x7f066c0b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0671100700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f066c06e620 0x7f066c14a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06708ff700 1 -- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f066c158eb0 msgr2=0x7f066c0b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06708ff700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f066c158eb0 0x7f066c0b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06708ff700 1 -- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f066c06e620 msgr2=0x7f066c14a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06708ff700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f066c06e620 0x7f066c14a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06708ff700 1 -- 172.21.15.38:0/2367959243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0664009a20 con 0x7f066c0aecc0 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0671100700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f066c06e620 0x7f066c14a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06708ff700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c0aecc0 0x7f066c0b0690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f065c00c810 tx=0x7f065c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06617fa700 1 -- 172.21.15.38:0/2367959243 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c005b30 con 0x7f066c0aecc0 2024-08-15T17:59:01.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06617fa700 1 -- 172.21.15.38:0/2367959243 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c005c90 con 0x7f066c0aecc0 2024-08-15T17:59:01.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f06617fa700 1 -- 172.21.15.38:0/2367959243 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c007340 con 0x7f066c0aecc0 2024-08-15T17:59:01.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0672102700 1 -- 172.21.15.38:0/2367959243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066c156720 con 0x7f066c0aecc0 2024-08-15T17:59:01.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0672102700 1 -- 172.21.15.38:0/2367959243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066c16d110 con 0x7f066c0aecc0 2024-08-15T17:59:01.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.837+0000 7f0672102700 1 -- 172.21.15.38:0/2367959243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066c0643a0 con 0x7f066c0aecc0 2024-08-15T17:59:01.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.841+0000 7f06617fa700 1 -- 172.21.15.38:0/2367959243 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f065c025430 con 0x7f066c0aecc0 2024-08-15T17:59:01.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.841+0000 7f06617fa700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0650060890 0x7f0650062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:01.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.841+0000 7f06617fa700 1 -- 172.21.15.38:0/2367959243 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f065c07d260 con 0x7f066c0aecc0 2024-08-15T17:59:01.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.841+0000 7f0671100700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0650060890 0x7f0650062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:01.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.841+0000 7f0671100700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0650060890 0x7f0650062d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0668009ce0 tx=0x7f0668008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:01.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:01.849+0000 7f06617fa700 1 -- 172.21.15.38:0/2367959243 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f065c04d710 con 0x7f066c0aecc0 2024-08-15T17:59:02.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.013+0000 7f0672102700 1 -- 172.21.15.38:0/2367959243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f066c152cb0 con 0x7f066c0aecc0 2024-08-15T17:59:02.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.013+0000 7f06617fa700 1 -- 172.21.15.38:0/2367959243 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f065c0076f0 con 0x7f066c0aecc0 2024-08-15T17:59:02.018 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:02.018 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:59:02.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 -- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0650060890 msgr2=0x7f0650062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0650060890 0x7f0650062d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0668009ce0 tx=0x7f0668008040 comp rx=0 tx=0).stop 2024-08-15T17:59:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 -- 172.21.15.38:0/2367959243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c0aecc0 msgr2=0x7f066c0b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c0aecc0 0x7f066c0b0690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f065c00c810 tx=0x7f065c007f30 comp rx=0 tx=0).stop 2024-08-15T17:59:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 -- 172.21.15.38:0/2367959243 shutdown_connections 2024-08-15T17:59:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0650060890 0x7f0650062d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f066c158eb0 0x7f066c0b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c0aecc0 0x7f066c0b0690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 --2- 172.21.15.38:0/2367959243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f066c06e620 0x7f066c14a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 -- 172.21.15.38:0/2367959243 >> 172.21.15.38:0/2367959243 conn(0x7f066c0678d0 msgr2=0x7f066c068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 -- 172.21.15.38:0/2367959243 shutdown_connections 2024-08-15T17:59:02.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.017+0000 7f0672102700 1 -- 172.21.15.38:0/2367959243 wait complete. 2024-08-15T17:59:02.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:58:34.079267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:58:34.621969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:59:02.033 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:59:02.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cd1d01700 1 Processor -- start 2024-08-15T17:59:02.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cd1d01700 1 -- start start 2024-08-15T17:59:02.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cd1d01700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1ccc1596d0 0x7f1ccc066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cd1d01700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ccc15dbc0 0x7f1ccc067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cd1d01700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ccc0cba90 con 0x7f1ccc159300 2024-08-15T17:59:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cd1d01700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ccc0b2b60 con 0x7f1ccc1596d0 2024-08-15T17:59:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cd1d01700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0b2940 con 0x7f1ccc15dbc0 2024-08-15T17:59:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cc3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1ccc1596d0 0x7f1ccc066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cc3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1ccc1596d0 0x7f1ccc066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38724/0 (socket says 172.21.15.38:38724) 2024-08-15T17:59:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cc3fff700 1 -- 172.21.15.38:0/218579549 learned_addr learned my addr 172.21.15.38:0/218579549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1ccbfff700 1 --2- 172.21.15.38:0/218579549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ccc15dbc0 0x7f1ccc067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cc3fff700 1 -- 172.21.15.38:0/218579549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ccc15dbc0 msgr2=0x7f1ccc067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cc3fff700 1 --2- 172.21.15.38:0/218579549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ccc15dbc0 0x7f1ccc067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1ccaffd700 1 -- 172.21.15.38:0/218579549 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617138593 0 0) 0x7f1ccc0cba90 con 0x7f1ccc159300 2024-08-15T17:59:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cc3fff700 1 -- 172.21.15.38:0/218579549 >> v1:172.21.15.104:6789/0 conn(0x7f1ccc159300 legacy=0x7f1ccc0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cc3fff700 1 -- 172.21.15.38:0/218579549 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc0cb160 con 0x7f1ccc1596d0 2024-08-15T17:59:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1cc3fff700 1 --2- 172.21.15.38:0/218579549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ccc1596d0 0x7f1ccc066e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40099e0 tx=0x7f1cb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fa5fdea7714421 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1ccaffd700 1 -- 172.21.15.38:0/218579549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400b820 con 0x7f1ccc1596d0 2024-08-15T17:59:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1ccaffd700 1 -- 172.21.15.38:0/218579549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb400b980 con 0x7f1ccc1596d0 2024-08-15T17:59:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.145+0000 7f1ccaffd700 1 -- 172.21.15.38:0/218579549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400bd20 con 0x7f1ccc1596d0 2024-08-15T17:59:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.149+0000 7f1cd1d01700 1 -- 172.21.15.38:0/218579549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ccc1596d0 msgr2=0x7f1ccc066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.149+0000 7f1cd1d01700 1 --2- 172.21.15.38:0/218579549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ccc1596d0 0x7f1ccc066e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40099e0 tx=0x7f1cb4009250 comp rx=0 tx=0).stop 2024-08-15T17:59:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.149+0000 7f1cd1d01700 1 -- 172.21.15.38:0/218579549 shutdown_connections 2024-08-15T17:59:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.149+0000 7f1cd1d01700 1 --2- 172.21.15.38:0/218579549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ccc15dbc0 0x7f1ccc067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.149+0000 7f1cd1d01700 1 --2- 172.21.15.38:0/218579549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ccc1596d0 0x7f1ccc066e00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.149+0000 7f1cd1d01700 1 -- 172.21.15.38:0/218579549 >> 172.21.15.38:0/218579549 conn(0x7f1ccc0682c0 msgr2=0x7f1ccc0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.149+0000 7f1cd1d01700 1 -- 172.21.15.38:0/218579549 shutdown_connections 2024-08-15T17:59:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.149+0000 7f1cd1d01700 1 -- 172.21.15.38:0/218579549 wait complete. 2024-08-15T17:59:02.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.149+0000 7f1cd1d01700 1 Processor -- start 2024-08-15T17:59:02.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.149+0000 7f1cd1d01700 1 -- start start 2024-08-15T17:59:02.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cd1d01700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ccc159300 0x7f1ccc0c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cd1d01700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ccc1596d0 0x7f1ccc0c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cd1d01700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc15dbc0 0x7f1ccc0c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cd1d01700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0c6d10 con 0x7f1ccc159300 2024-08-15T17:59:02.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cd1d01700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0c3cb0 con 0x7f1ccc15dbc0 2024-08-15T17:59:02.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cd1d01700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0c3f90 con 0x7f1ccc1596d0 2024-08-15T17:59:02.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1ccb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ccc159300 0x7f1ccc0c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1ccb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ccc159300 0x7f1ccc0c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38732/0 (socket says 172.21.15.38:38732) 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1ccb7fe700 1 -- 172.21.15.38:0/1700344763 learned_addr learned my addr 172.21.15.38:0/1700344763 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1ccbfff700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc15dbc0 0x7f1ccc0c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cc3fff700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ccc1596d0 0x7f1ccc0c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1ccb7fe700 1 -- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ccc1596d0 msgr2=0x7f1ccc0c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1ccb7fe700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ccc1596d0 0x7f1ccc0c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1ccb7fe700 1 -- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc15dbc0 msgr2=0x7f1ccc0c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1ccb7fe700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc15dbc0 0x7f1ccc0c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1ccb7fe700 1 -- 172.21.15.38:0/1700344763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb40094f0 con 0x7f1ccc159300 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1ccbfff700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc15dbc0 0x7f1ccc0c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cc3fff700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ccc1596d0 0x7f1ccc0c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1ccb7fe700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ccc159300 0x7f1ccc0c67d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00c530 tx=0x7f1cbc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:02.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cc37fe700 1 -- 172.21.15.38:0/1700344763 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc011590 con 0x7f1ccc159300 2024-08-15T17:59:02.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cc37fe700 1 -- 172.21.15.38:0/1700344763 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc0116f0 con 0x7f1ccc159300 2024-08-15T17:59:02.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cd1d01700 1 -- 172.21.15.38:0/1700344763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ccc0c4270 con 0x7f1ccc159300 2024-08-15T17:59:02.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cc37fe700 1 -- 172.21.15.38:0/1700344763 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc011ab0 con 0x7f1ccc159300 2024-08-15T17:59:02.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cd1d01700 1 -- 172.21.15.38:0/1700344763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ccc0c4830 con 0x7f1ccc159300 2024-08-15T17:59:02.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cd1d01700 1 -- 172.21.15.38:0/1700344763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ccc0645c0 con 0x7f1ccc159300 2024-08-15T17:59:02.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.153+0000 7f1cc37fe700 1 -- 172.21.15.38:0/1700344763 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1cbc01b3b0 con 0x7f1ccc159300 2024-08-15T17:59:02.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.157+0000 7f1cc37fe700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1cac060890 0x7f1cac062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.157+0000 7f1cc37fe700 1 -- 172.21.15.38:0/1700344763 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1cbc07c130 con 0x7f1ccc159300 2024-08-15T17:59:02.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.157+0000 7f1cc3fff700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1cac060890 0x7f1cac062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.157+0000 7f1cc3fff700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1cac060890 0x7f1cac062d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40099e0 tx=0x7f1cb4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:02.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.161+0000 7f1cc37fe700 1 -- 172.21.15.38:0/1700344763 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1cbc04c580 con 0x7f1ccc159300 2024-08-15T17:59:02.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.325+0000 7f1cd1d01700 1 -- 172.21.15.38:0/1700344763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ccc0caac0 con 0x7f1ccc159300 2024-08-15T17:59:02.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.325+0000 7f1cc37fe700 1 -- 172.21.15.38:0/1700344763 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1cbc017020 con 0x7f1ccc159300 2024-08-15T17:59:02.330 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T17:59:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 -- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1cac060890 msgr2=0x7f1cac062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1cac060890 0x7f1cac062d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40099e0 tx=0x7f1cb4017000 comp rx=0 tx=0).stop 2024-08-15T17:59:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 -- 172.21.15.38:0/1700344763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ccc159300 msgr2=0x7f1ccc0c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ccc159300 0x7f1ccc0c67d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00c530 tx=0x7f1cbc00c8f0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 -- 172.21.15.38:0/1700344763 shutdown_connections 2024-08-15T17:59:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1cac060890 0x7f1cac062d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc15dbc0 0x7f1ccc0c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ccc1596d0 0x7f1ccc0c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 --2- 172.21.15.38:0/1700344763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ccc159300 0x7f1ccc0c67d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 -- 172.21.15.38:0/1700344763 >> 172.21.15.38:0/1700344763 conn(0x7f1ccc0682c0 msgr2=0x7f1ccc0b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 -- 172.21.15.38:0/1700344763 shutdown_connections 2024-08-15T17:59:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.329+0000 7f1cd1d01700 1 -- 172.21.15.38:0/1700344763 wait complete. 2024-08-15T17:59:02.344 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:59:02.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.457+0000 7f48ea178700 1 Processor -- start 2024-08-15T17:59:02.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.457+0000 7f48ea178700 1 -- start start 2024-08-15T17:59:02.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.457+0000 7f48ea178700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48e4159480 0x7f48e40cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.457+0000 7f48ea178700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e40ce210 con 0x7f48e41590b0 2024-08-15T17:59:02.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.457+0000 7f48ea178700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e40cb440 con 0x7f48e4067510 2024-08-15T17:59:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.457+0000 7f48ea178700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40cba00 con 0x7f48e4159480 2024-08-15T17:59:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.457+0000 7f48e9176700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f48e41590b0 0x7f48e4066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41064/0 (socket says 172.21.15.38:41064) 2024-08-15T17:59:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.457+0000 7f48e9176700 1 -- 172.21.15.38:0/3918702526 learned_addr learned my addr 172.21.15.38:0/3918702526 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.457+0000 7f48e9977700 1 --2- 172.21.15.38:0/3918702526 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48e4159480 0x7f48e40cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48dbfff700 1 -- 172.21.15.38:0/3918702526 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097192549 0 0) 0x7f48e40ce210 con 0x7f48e41590b0 2024-08-15T17:59:02.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48dbfff700 1 -- 172.21.15.38:0/3918702526 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48d0003570 con 0x7f48e41590b0 2024-08-15T17:59:02.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48dbfff700 1 -- 172.21.15.38:0/3918702526 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300092639 0 0) 0x7f48e40cb440 con 0x7f48e4067510 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48dbfff700 1 -- 172.21.15.38:0/3918702526 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e40ce210 con 0x7f48e4067510 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48e9977700 1 -- 172.21.15.38:0/3918702526 >> v1:172.21.15.104:6789/0 conn(0x7f48e4067510 legacy=0x7f48e40ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48e9977700 1 -- 172.21.15.38:0/3918702526 >> v1:172.21.15.38:6789/0 conn(0x7f48e41590b0 legacy=0x7f48e4066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48e9977700 1 -- 172.21.15.38:0/3918702526 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e40cb720 con 0x7f48e4159480 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48e9977700 1 --2- 172.21.15.38:0/3918702526 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48e4159480 0x7f48e40cedb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009e90 tx=0x7f48dc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d694911f0d383c8e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48dbfff700 1 -- 172.21.15.38:0/3918702526 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00e070 con 0x7f48e4159480 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48dbfff700 1 -- 172.21.15.38:0/3918702526 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc00bb80 con 0x7f48e4159480 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48dbfff700 1 -- 172.21.15.38:0/3918702526 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc012430 con 0x7f48e4159480 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 -- 172.21.15.38:0/3918702526 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48e4159480 msgr2=0x7f48e40cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 --2- 172.21.15.38:0/3918702526 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48e4159480 0x7f48e40cedb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009e90 tx=0x7f48dc009780 comp rx=0 tx=0).stop 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 -- 172.21.15.38:0/3918702526 shutdown_connections 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 --2- 172.21.15.38:0/3918702526 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48e4159480 0x7f48e40cedb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 -- 172.21.15.38:0/3918702526 >> 172.21.15.38:0/3918702526 conn(0x7f48e40d0b90 msgr2=0x7f48e40ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:02.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 -- 172.21.15.38:0/3918702526 shutdown_connections 2024-08-15T17:59:02.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 -- 172.21.15.38:0/3918702526 wait complete. 2024-08-15T17:59:02.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 Processor -- start 2024-08-15T17:59:02.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 -- start start 2024-08-15T17:59:02.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48e4067510 0x7f48e4150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48e41590b0 0x7f48e41558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48e4159480 0x7f48e415dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e415c350 con 0x7f48e4159480 2024-08-15T17:59:02.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e415e710 con 0x7f48e4067510 2024-08-15T17:59:02.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.461+0000 7f48ea178700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e415ba20 con 0x7f48e41590b0 2024-08-15T17:59:02.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e9977700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48e4159480 0x7f48e415dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e9977700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48e4159480 0x7f48e415dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38742/0 (socket says 172.21.15.38:38742) 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e9977700 1 -- 172.21.15.38:0/1026292339 learned_addr learned my addr 172.21.15.38:0/1026292339 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e8975700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48e41590b0 0x7f48e41558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e9176700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48e4067510 0x7f48e4150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e9977700 1 -- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48e41590b0 msgr2=0x7f48e41558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e9977700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48e41590b0 0x7f48e41558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e9977700 1 -- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48e4067510 msgr2=0x7f48e4150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e9977700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48e4067510 0x7f48e4150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e9977700 1 -- 172.21.15.38:0/1026292339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc009a20 con 0x7f48e4159480 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e9176700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48e4067510 0x7f48e4150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48e9977700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48e4159480 0x7f48e415dcc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f48dc013000 tx=0x7f48dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48d97fa700 1 -- 172.21.15.38:0/1026292339 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc027470 con 0x7f48e4159480 2024-08-15T17:59:02.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48d97fa700 1 -- 172.21.15.38:0/1026292339 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc0275f0 con 0x7f48e4159480 2024-08-15T17:59:02.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48ea178700 1 -- 172.21.15.38:0/1026292339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e415bd00 con 0x7f48e4159480 2024-08-15T17:59:02.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48d97fa700 1 -- 172.21.15.38:0/1026292339 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc0279b0 con 0x7f48e4159480 2024-08-15T17:59:02.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48ea178700 1 -- 172.21.15.38:0/1026292339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e4155e20 con 0x7f48e4159480 2024-08-15T17:59:02.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48d97fa700 1 -- 172.21.15.38:0/1026292339 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f48dc024070 con 0x7f48e4159480 2024-08-15T17:59:02.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48ea178700 1 -- 172.21.15.38:0/1026292339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e415c500 con 0x7f48e4159480 2024-08-15T17:59:02.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.465+0000 7f48d97fa700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f48c8060860 0x7f48c8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.469+0000 7f48d97fa700 1 -- 172.21.15.38:0/1026292339 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f48dc07f370 con 0x7f48e4159480 2024-08-15T17:59:02.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.469+0000 7f48e9176700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f48c8060860 0x7f48c8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.469+0000 7f48e9176700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f48c8060860 0x7f48c8062d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f48d40058d0 tx=0x7f48d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:02.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.473+0000 7f48d97fa700 1 -- 172.21.15.38:0/1026292339 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f48dc04f760 con 0x7f48e4159480 2024-08-15T17:59:02.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.637+0000 7f48ea178700 1 -- 172.21.15.38:0/1026292339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48e4064330 con 0x7f48e4159480 2024-08-15T17:59:02.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.637+0000 7f48d97fa700 1 -- 172.21.15.38:0/1026292339 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f48dc052d80 con 0x7f48e4159480 2024-08-15T17:59:02.643 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T17:59:02.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 -- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f48c8060860 msgr2=0x7f48c8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f48c8060860 0x7f48c8062d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f48d40058d0 tx=0x7f48d4009040 comp rx=0 tx=0).stop 2024-08-15T17:59:02.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 -- 172.21.15.38:0/1026292339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48e4159480 msgr2=0x7f48e415dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48e4159480 0x7f48e415dcc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f48dc013000 tx=0x7f48dc019000 comp rx=0 tx=0).stop 2024-08-15T17:59:02.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 -- 172.21.15.38:0/1026292339 shutdown_connections 2024-08-15T17:59:02.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f48c8060860 0x7f48c8062d10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48e4159480 0x7f48e415dcc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48e41590b0 0x7f48e41558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 --2- 172.21.15.38:0/1026292339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48e4067510 0x7f48e4150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 -- 172.21.15.38:0/1026292339 >> 172.21.15.38:0/1026292339 conn(0x7f48e40d0b90 msgr2=0x7f48e40b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:02.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 -- 172.21.15.38:0/1026292339 shutdown_connections 2024-08-15T17:59:02.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.641+0000 7f48ea178700 1 -- 172.21.15.38:0/1026292339 wait complete. 2024-08-15T17:59:02.655 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:59:02.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.773+0000 7f3e17380700 1 Processor -- start 2024-08-15T17:59:02.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.773+0000 7f3e17380700 1 -- start start 2024-08-15T17:59:02.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.773+0000 7f3e17380700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3e10159300 0x7f3e100b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.773+0000 7f3e17380700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3e101596d0 0x7f3e10066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.773+0000 7f3e17380700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e100cba90 con 0x7f3e1015dbc0 2024-08-15T17:59:02.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.773+0000 7f3e17380700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e100b2b60 con 0x7f3e101596d0 2024-08-15T17:59:02.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.773+0000 7f3e17380700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e100cb160 con 0x7f3e10159300 2024-08-15T17:59:02.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.773+0000 7f3e1491b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3e101596d0 0x7f3e10066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.773+0000 7f3e1491b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3e101596d0 0x7f3e10066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38750/0 (socket says 172.21.15.38:38750) 2024-08-15T17:59:02.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.773+0000 7f3e1491b700 1 -- 172.21.15.38:0/448490656 learned_addr learned my addr 172.21.15.38:0/448490656 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:02.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.773+0000 7f3e1511c700 1 --2- 172.21.15.38:0/448490656 >> v2:172.21.15.104:3300/0 conn(0x7f3e10159300 0x7f3e100b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e1491b700 1 -- 172.21.15.38:0/448490656 >> v2:172.21.15.104:3300/0 conn(0x7f3e10159300 msgr2=0x7f3e100b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:02.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e1491b700 1 --2- 172.21.15.38:0/448490656 >> v2:172.21.15.104:3300/0 conn(0x7f3e10159300 0x7f3e100b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e1491b700 1 -- 172.21.15.38:0/448490656 >> v1:172.21.15.104:6789/0 conn(0x7f3e1015dbc0 legacy=0x7f3e100afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e1491b700 1 -- 172.21.15.38:0/448490656 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e100cb440 con 0x7f3e101596d0 2024-08-15T17:59:02.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e1491b700 1 --2- 172.21.15.38:0/448490656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e101596d0 0x7f3e10066e00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3e10089460 tx=0x7f3e00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1996e25e1b7426b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e07fff700 1 -- 172.21.15.38:0/448490656 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e00009e70 con 0x7f3e101596d0 2024-08-15T17:59:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e07fff700 1 -- 172.21.15.38:0/448490656 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0000b860 con 0x7f3e101596d0 2024-08-15T17:59:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e07fff700 1 -- 172.21.15.38:0/448490656 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0000bc00 con 0x7f3e101596d0 2024-08-15T17:59:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e17380700 1 -- 172.21.15.38:0/448490656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e101596d0 msgr2=0x7f3e10066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e17380700 1 --2- 172.21.15.38:0/448490656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e101596d0 0x7f3e10066e00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3e10089460 tx=0x7f3e00009250 comp rx=0 tx=0).stop 2024-08-15T17:59:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e17380700 1 -- 172.21.15.38:0/448490656 shutdown_connections 2024-08-15T17:59:02.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e17380700 1 --2- 172.21.15.38:0/448490656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e101596d0 0x7f3e10066e00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e17380700 1 --2- 172.21.15.38:0/448490656 >> v2:172.21.15.104:3300/0 conn(0x7f3e10159300 0x7f3e100b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e17380700 1 -- 172.21.15.38:0/448490656 >> 172.21.15.38:0/448490656 conn(0x7f3e100682c0 msgr2=0x7f3e100686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:02.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e17380700 1 -- 172.21.15.38:0/448490656 shutdown_connections 2024-08-15T17:59:02.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.777+0000 7f3e17380700 1 -- 172.21.15.38:0/448490656 wait complete. 2024-08-15T17:59:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e17380700 1 Processor -- start 2024-08-15T17:59:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e17380700 1 -- start start 2024-08-15T17:59:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e17380700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e10159300 0x7f3e101695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e17380700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e101596d0 0x7f3e100c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e17380700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e1015dbc0 0x7f3e100c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e17380700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e100c3cb0 con 0x7f3e10159300 2024-08-15T17:59:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e17380700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e10169e20 con 0x7f3e1015dbc0 2024-08-15T17:59:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e17380700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1016a100 con 0x7f3e101596d0 2024-08-15T17:59:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1511c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e10159300 0x7f3e101695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1511c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e10159300 0x7f3e101695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38766/0 (socket says 172.21.15.38:38766) 2024-08-15T17:59:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1511c700 1 -- 172.21.15.38:0/203387922 learned_addr learned my addr 172.21.15.38:0/203387922 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1591d700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e1015dbc0 0x7f3e100c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1491b700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e101596d0 0x7f3e100c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1511c700 1 -- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e101596d0 msgr2=0x7f3e100c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1511c700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e101596d0 0x7f3e100c69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1511c700 1 -- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e1015dbc0 msgr2=0x7f3e100c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1511c700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e1015dbc0 0x7f3e100c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1511c700 1 -- 172.21.15.38:0/203387922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e000094f0 con 0x7f3e10159300 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1491b700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e101596d0 0x7f3e100c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1591d700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e1015dbc0 0x7f3e100c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e1511c700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e10159300 0x7f3e101695f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009e50 tx=0x7f3e0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e057fa700 1 -- 172.21.15.38:0/203387922 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00cc20 con 0x7f3e10159300 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e057fa700 1 -- 172.21.15.38:0/203387922 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c00f040 con 0x7f3e10159300 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e17380700 1 -- 172.21.15.38:0/203387922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1016a3e0 con 0x7f3e10159300 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.781+0000 7f3e057fa700 1 -- 172.21.15.38:0/203387922 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c014480 con 0x7f3e10159300 2024-08-15T17:59:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.785+0000 7f3e17380700 1 -- 172.21.15.38:0/203387922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e10169b30 con 0x7f3e10159300 2024-08-15T17:59:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.785+0000 7f3e17380700 1 -- 172.21.15.38:0/203387922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e100c40f0 con 0x7f3e10159300 2024-08-15T17:59:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.785+0000 7f3e057fa700 1 -- 172.21.15.38:0/203387922 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3e0c0147f0 con 0x7f3e10159300 2024-08-15T17:59:02.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.789+0000 7f3e057fa700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3df4060890 0x7f3df4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:02.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.789+0000 7f3e057fa700 1 -- 172.21.15.38:0/203387922 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3e0c07d520 con 0x7f3e10159300 2024-08-15T17:59:02.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.789+0000 7f3e1491b700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3df4060890 0x7f3df4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:02.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.789+0000 7f3e1491b700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3df4060890 0x7f3df4062d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3e10089460 tx=0x7f3e00017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:02.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.793+0000 7f3e057fa700 1 -- 172.21.15.38:0/203387922 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3e0c04d970 con 0x7f3e10159300 2024-08-15T17:59:02.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.953+0000 7f3e17380700 1 -- 172.21.15.38:0/203387922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e100645c0 con 0x7f3e10159300 2024-08-15T17:59:02.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.953+0000 7f3e057fa700 1 -- 172.21.15.38:0/203387922 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e0c07be60 con 0x7f3e10159300 2024-08-15T17:59:02.956 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 -- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3df4060890 msgr2=0x7f3df4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3df4060890 0x7f3df4062d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3e10089460 tx=0x7f3e00017000 comp rx=0 tx=0).stop 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 -- 172.21.15.38:0/203387922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e10159300 msgr2=0x7f3e101695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e10159300 0x7f3e101695f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009e50 tx=0x7f3e0c00b040 comp rx=0 tx=0).stop 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 -- 172.21.15.38:0/203387922 shutdown_connections 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3df4060890 0x7f3df4062d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e1015dbc0 0x7f3e100c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e101596d0 0x7f3e100c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 --2- 172.21.15.38:0/203387922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e10159300 0x7f3e101695f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 -- 172.21.15.38:0/203387922 >> 172.21.15.38:0/203387922 conn(0x7f3e100682c0 msgr2=0x7f3e101585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 -- 172.21.15.38:0/203387922 shutdown_connections 2024-08-15T17:59:02.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:02.957+0000 7f3e17380700 1 -- 172.21.15.38:0/203387922 wait complete. 2024-08-15T17:59:02.975 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:59:03.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a9ccb700 1 Processor -- start 2024-08-15T17:59:03.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a9ccb700 1 -- start start 2024-08-15T17:59:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a9ccb700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f04a4068840 0x7f04a4068c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a9ccb700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04a40ae8e0 0x7f04a4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a9ccb700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f04a40aed40 0x7f04a415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a9ccb700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a40b0350 con 0x7f04a4068840 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a9ccb700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a40af7d0 con 0x7f04a40aed40 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a9ccb700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40afab0 con 0x7f04a40ae8e0 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a8cc9700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f04a4068840 0x7f04a4068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a8cc9700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f04a4068840 0x7f04a4068c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38782/0 (socket says 172.21.15.38:38782) 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a8cc9700 1 -- 172.21.15.38:0/3779795349 learned_addr learned my addr 172.21.15.38:0/3779795349 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a3fff700 1 --2- 172.21.15.38:0/3779795349 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04a40ae8e0 0x7f04a4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a94ca700 1 --2- 172.21.15.38:0/3779795349 >> v2:172.21.15.104:3300/0 conn(0x7f04a40aed40 0x7f04a415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.081+0000 7f04a8cc9700 1 -- 172.21.15.38:0/3779795349 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04a40ae8e0 msgr2=0x7f04a4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a8cc9700 1 --2- 172.21.15.38:0/3779795349 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04a40ae8e0 0x7f04a4159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a8cc9700 1 -- 172.21.15.38:0/3779795349 >> v2:172.21.15.104:3300/0 conn(0x7f04a40aed40 msgr2=0x7f04a415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a8cc9700 1 --2- 172.21.15.38:0/3779795349 >> v2:172.21.15.104:3300/0 conn(0x7f04a40aed40 0x7f04a415a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a8cc9700 1 -- 172.21.15.38:0/3779795349 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a40afd90 con 0x7f04a4068840 2024-08-15T17:59:03.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a8cc9700 1 --2- 172.21.15.38:0/3779795349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a4068840 0x7f04a4068c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f04a4069150 tx=0x7f0494009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd09ed06b92ceeb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:03.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a37fe700 1 -- 172.21.15.38:0/3779795349 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049400b860 con 0x7f04a4068840 2024-08-15T17:59:03.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a37fe700 1 -- 172.21.15.38:0/3779795349 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049400b9c0 con 0x7f04a4068840 2024-08-15T17:59:03.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a37fe700 1 -- 172.21.15.38:0/3779795349 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049400bd60 con 0x7f04a4068840 2024-08-15T17:59:03.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a9ccb700 1 -- 172.21.15.38:0/3779795349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a4068840 msgr2=0x7f04a4068c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a9ccb700 1 --2- 172.21.15.38:0/3779795349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a4068840 0x7f04a4068c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f04a4069150 tx=0x7f0494009250 comp rx=0 tx=0).stop 2024-08-15T17:59:03.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a9ccb700 1 -- 172.21.15.38:0/3779795349 shutdown_connections 2024-08-15T17:59:03.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a9ccb700 1 --2- 172.21.15.38:0/3779795349 >> v2:172.21.15.104:3300/0 conn(0x7f04a40aed40 0x7f04a415a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a9ccb700 1 --2- 172.21.15.38:0/3779795349 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04a40ae8e0 0x7f04a4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a9ccb700 1 --2- 172.21.15.38:0/3779795349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a4068840 0x7f04a4068c10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a9ccb700 1 -- 172.21.15.38:0/3779795349 >> 172.21.15.38:0/3779795349 conn(0x7f04a406d800 msgr2=0x7f04a406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:03.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a9ccb700 1 -- 172.21.15.38:0/3779795349 shutdown_connections 2024-08-15T17:59:03.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.085+0000 7f04a9ccb700 1 -- 172.21.15.38:0/3779795349 wait complete. 2024-08-15T17:59:03.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a9ccb700 1 Processor -- start 2024-08-15T17:59:03.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a9ccb700 1 -- start start 2024-08-15T17:59:03.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a9ccb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04a4068840 0x7f04a40aa040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a9ccb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a40ae8e0 0x7f04a40c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a9ccb700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04a40aed40 0x7f04a40c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a9ccb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40c2020 con 0x7f04a40ae8e0 2024-08-15T17:59:03.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a9ccb700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40c22f0 con 0x7f04a4068840 2024-08-15T17:59:03.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a9ccb700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40c25d0 con 0x7f04a40aed40 2024-08-15T17:59:03.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a40ae8e0 0x7f04a40c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a40ae8e0 0x7f04a40c15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38798/0 (socket says 172.21.15.38:38798) 2024-08-15T17:59:03.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a8cc9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04a4068840 0x7f04a40aa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.089+0000 7f04a3fff700 1 -- 172.21.15.38:0/383010654 learned_addr learned my addr 172.21.15.38:0/383010654 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:03.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a94ca700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04a40aed40 0x7f04a40c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a3fff700 1 -- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04a40aed40 msgr2=0x7f04a40c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a3fff700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04a40aed40 0x7f04a40c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a3fff700 1 -- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04a4068840 msgr2=0x7f04a40aa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a3fff700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04a4068840 0x7f04a40aa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a3fff700 1 -- 172.21.15.38:0/383010654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940094f0 con 0x7f04a40ae8e0 2024-08-15T17:59:03.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a3fff700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a40ae8e0 0x7f04a40c15a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f049800fc10 tx=0x7f0498008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:03.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a0ff9700 1 -- 172.21.15.38:0/383010654 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04980078f0 con 0x7f04a40ae8e0 2024-08-15T17:59:03.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a0ff9700 1 -- 172.21.15.38:0/383010654 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0498007a50 con 0x7f04a40ae8e0 2024-08-15T17:59:03.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a9ccb700 1 -- 172.21.15.38:0/383010654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a40c28b0 con 0x7f04a40ae8e0 2024-08-15T17:59:03.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a0ff9700 1 -- 172.21.15.38:0/383010654 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498007df0 con 0x7f04a40ae8e0 2024-08-15T17:59:03.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a9ccb700 1 -- 172.21.15.38:0/383010654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a40c2e70 con 0x7f04a40ae8e0 2024-08-15T17:59:03.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.093+0000 7f04a9ccb700 1 -- 172.21.15.38:0/383010654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a40638f0 con 0x7f04a40ae8e0 2024-08-15T17:59:03.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.097+0000 7f04a0ff9700 1 -- 172.21.15.38:0/383010654 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f049801a670 con 0x7f04a40ae8e0 2024-08-15T17:59:03.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.097+0000 7f04a0ff9700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0490060860 0x7f0490062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.097+0000 7f04a0ff9700 1 -- 172.21.15.38:0/383010654 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f049807f710 con 0x7f04a40ae8e0 2024-08-15T17:59:03.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.097+0000 7f04a8cc9700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0490060860 0x7f0490062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.097+0000 7f04a8cc9700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0490060860 0x7f0490062d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f04a4069150 tx=0x7f0494017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:03.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.105+0000 7f04a0ff9700 1 -- 172.21.15.38:0/383010654 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f049804fc10 con 0x7f04a40ae8e0 2024-08-15T17:59:03.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 -- 172.21.15.38:0/383010654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04a40b2100 con 0x7f04a40ae8e0 2024-08-15T17:59:03.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a0ff9700 1 -- 172.21.15.38:0/383010654 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f0498015090 con 0x7f04a40ae8e0 2024-08-15T17:59:03.268 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:03.268 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:59:03.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 -- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0490060860 msgr2=0x7f0490062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0490060860 0x7f0490062d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f04a4069150 tx=0x7f0494017000 comp rx=0 tx=0).stop 2024-08-15T17:59:03.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 -- 172.21.15.38:0/383010654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a40ae8e0 msgr2=0x7f04a40c15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a40ae8e0 0x7f04a40c15a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f049800fc10 tx=0x7f0498008040 comp rx=0 tx=0).stop 2024-08-15T17:59:03.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 -- 172.21.15.38:0/383010654 shutdown_connections 2024-08-15T17:59:03.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0490060860 0x7f0490062d10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04a40aed40 0x7f04a40c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a40ae8e0 0x7f04a40c15a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 --2- 172.21.15.38:0/383010654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04a4068840 0x7f04a40aa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 -- 172.21.15.38:0/383010654 >> 172.21.15.38:0/383010654 conn(0x7f04a406d800 msgr2=0x7f04a40a5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:03.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 -- 172.21.15.38:0/383010654 shutdown_connections 2024-08-15T17:59:03.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.265+0000 7f04a9ccb700 1 -- 172.21.15.38:0/383010654 wait complete. 2024-08-15T17:59:03.283 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:58:34.079267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:58:34.621969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:59:03.283 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:59:03.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.389+0000 7fa2ec8e7700 1 Processor -- start 2024-08-15T17:59:03.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.389+0000 7fa2ec8e7700 1 -- start start 2024-08-15T17:59:03.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2ec8e7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa2e8159480 0x7fa2e80cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2ec8e7700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e80ce210 con 0x7fa2e8067510 2024-08-15T17:59:03.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2ec8e7700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e80cb440 con 0x7fa2e81590b0 2024-08-15T17:59:03.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2ec8e7700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e80cba00 con 0x7fa2e8159480 2024-08-15T17:59:03.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e659c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa2e8067510 0x7fa2e80ce6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41074/0 (socket says 172.21.15.38:41074) 2024-08-15T17:59:03.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e659c700 1 -- 172.21.15.38:0/4128716873 learned_addr learned my addr 172.21.15.38:0/4128716873 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:03.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e759e700 1 --2- 172.21.15.38:0/4128716873 >> v2:172.21.15.104:3300/0 conn(0x7fa2e8159480 0x7fa2e80cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e5d9b700 1 -- 172.21.15.38:0/4128716873 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648770260 0 0) 0x7fa2e80ce210 con 0x7fa2e8067510 2024-08-15T17:59:03.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e5d9b700 1 -- 172.21.15.38:0/4128716873 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2d0003570 con 0x7fa2e8067510 2024-08-15T17:59:03.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e5d9b700 1 -- 172.21.15.38:0/4128716873 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635990740 0 0) 0x7fa2e80cb440 con 0x7fa2e81590b0 2024-08-15T17:59:03.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e5d9b700 1 -- 172.21.15.38:0/4128716873 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e80ce210 con 0x7fa2e81590b0 2024-08-15T17:59:03.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e759e700 1 -- 172.21.15.38:0/4128716873 >> v1:172.21.15.104:6789/0 conn(0x7fa2e81590b0 legacy=0x7fa2e8066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e759e700 1 -- 172.21.15.38:0/4128716873 >> v1:172.21.15.38:6789/0 conn(0x7fa2e8067510 legacy=0x7fa2e80ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e5d9b700 1 -- 172.21.15.38:0/4128716873 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4122984037 0 0) 0x7fa2d0003570 con 0x7fa2e8067510 2024-08-15T17:59:03.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e759e700 1 -- 172.21.15.38:0/4128716873 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80cb720 con 0x7fa2e8159480 2024-08-15T17:59:03.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e759e700 1 --2- 172.21.15.38:0/4128716873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2e8159480 0x7fa2e80cedb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009e60 tx=0x7fa2e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e36a6ecd337aa9aa server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:03.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e5d9b700 1 -- 172.21.15.38:0/4128716873 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e000e070 con 0x7fa2e8159480 2024-08-15T17:59:03.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e5d9b700 1 -- 172.21.15.38:0/4128716873 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e000bab0 con 0x7fa2e8159480 2024-08-15T17:59:03.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2e5d9b700 1 -- 172.21.15.38:0/4128716873 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e000be30 con 0x7fa2e8159480 2024-08-15T17:59:03.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/4128716873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2e8159480 msgr2=0x7fa2e80cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2ec8e7700 1 --2- 172.21.15.38:0/4128716873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2e8159480 0x7fa2e80cedb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009e60 tx=0x7fa2e0009780 comp rx=0 tx=0).stop 2024-08-15T17:59:03.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/4128716873 shutdown_connections 2024-08-15T17:59:03.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2ec8e7700 1 --2- 172.21.15.38:0/4128716873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2e8159480 0x7fa2e80cedb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/4128716873 >> 172.21.15.38:0/4128716873 conn(0x7fa2e80d0b90 msgr2=0x7fa2e80ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:03.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/4128716873 shutdown_connections 2024-08-15T17:59:03.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.393+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/4128716873 wait complete. 2024-08-15T17:59:03.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2ec8e7700 1 Processor -- start 2024-08-15T17:59:03.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2ec8e7700 1 -- start start 2024-08-15T17:59:03.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2ec8e7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2e8067510 0x7fa2e81575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2ec8e7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2e81590b0 0x7fa2e8157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2ec8e7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa2e8159480 0x7fa2e8169460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2ec8e7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e816a630 con 0x7fa2e81590b0 2024-08-15T17:59:03.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2ec8e7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e816aed0 con 0x7fa2e8067510 2024-08-15T17:59:03.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2ec8e7700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8158350 con 0x7fa2e8159480 2024-08-15T17:59:03.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2e659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2e81590b0 0x7fa2e8157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2e659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2e81590b0 0x7fa2e8157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38812/0 (socket says 172.21.15.38:38812) 2024-08-15T17:59:03.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2e659c700 1 -- 172.21.15.38:0/3835401130 learned_addr learned my addr 172.21.15.38:0/3835401130 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:03.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2e6d9d700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2e8067510 0x7fa2e81575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.397+0000 7fa2e759e700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa2e8159480 0x7fa2e8169460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2e659c700 1 -- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa2e8159480 msgr2=0x7fa2e8169460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2e659c700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa2e8159480 0x7fa2e8169460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2e659c700 1 -- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2e8067510 msgr2=0x7fa2e81575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2e659c700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2e8067510 0x7fa2e81575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2e659c700 1 -- 172.21.15.38:0/3835401130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e0009a20 con 0x7fa2e81590b0 2024-08-15T17:59:03.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2e6d9d700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2e8067510 0x7fa2e81575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:03.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2e659c700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2e81590b0 0x7fa2e8157b20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c5f0 tx=0x7fa2d8007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:03.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2ceffd700 1 -- 172.21.15.38:0/3835401130 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8007360 con 0x7fa2e81590b0 2024-08-15T17:59:03.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2ceffd700 1 -- 172.21.15.38:0/3835401130 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d80074c0 con 0x7fa2e81590b0 2024-08-15T17:59:03.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/3835401130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e816a880 con 0x7fa2e81590b0 2024-08-15T17:59:03.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2ceffd700 1 -- 172.21.15.38:0/3835401130 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8004830 con 0x7fa2e81590b0 2024-08-15T17:59:03.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/3835401130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e8158060 con 0x7fa2e81590b0 2024-08-15T17:59:03.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/3835401130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e816b060 con 0x7fa2e81590b0 2024-08-15T17:59:03.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.401+0000 7fa2ceffd700 1 -- 172.21.15.38:0/3835401130 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa2d8004c20 con 0x7fa2e81590b0 2024-08-15T17:59:03.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.405+0000 7fa2ceffd700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa2d4060860 0x7fa2d4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.405+0000 7fa2ceffd700 1 -- 172.21.15.38:0/3835401130 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa2d807c860 con 0x7fa2e81590b0 2024-08-15T17:59:03.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.405+0000 7fa2e6d9d700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa2d4060860 0x7fa2d4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.405+0000 7fa2e6d9d700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa2d4060860 0x7fa2d4062d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc007fd0 tx=0x7fa2dc007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:03.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.413+0000 7fa2ceffd700 1 -- 172.21.15.38:0/3835401130 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa2d804cd30 con 0x7fa2e81590b0 2024-08-15T17:59:03.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.577+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/3835401130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2e8064330 con 0x7fa2e81590b0 2024-08-15T17:59:03.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.577+0000 7fa2ceffd700 1 -- 172.21.15.38:0/3835401130 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2d8050350 con 0x7fa2e81590b0 2024-08-15T17:59:03.581 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T17:59:03.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa2d4060860 msgr2=0x7fa2d4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa2d4060860 0x7fa2d4062d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc007fd0 tx=0x7fa2dc007d50 comp rx=0 tx=0).stop 2024-08-15T17:59:03.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/3835401130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2e81590b0 msgr2=0x7fa2e8157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2e81590b0 0x7fa2e8157b20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c5f0 tx=0x7fa2d8007b00 comp rx=0 tx=0).stop 2024-08-15T17:59:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/3835401130 shutdown_connections 2024-08-15T17:59:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa2d4060860 0x7fa2d4062d10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa2e8159480 0x7fa2e8169460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2e81590b0 0x7fa2e8157b20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 --2- 172.21.15.38:0/3835401130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2e8067510 0x7fa2e81575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/3835401130 >> 172.21.15.38:0/3835401130 conn(0x7fa2e80d0b90 msgr2=0x7fa2e80c67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/3835401130 shutdown_connections 2024-08-15T17:59:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.581+0000 7fa2ec8e7700 1 -- 172.21.15.38:0/3835401130 wait complete. 2024-08-15T17:59:03.595 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:59:03.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.705+0000 7f16ac1f6700 1 Processor -- start 2024-08-15T17:59:03.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.705+0000 7f16ac1f6700 1 -- start start 2024-08-15T17:59:03.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.705+0000 7f16ac1f6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f16a40b2210 0x7f16a40b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.705+0000 7f16ac1f6700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a4159ce0 con 0x7f16a40b5040 2024-08-15T17:59:03.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.705+0000 7f16ac1f6700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a4159fc0 con 0x7f16a40cc840 2024-08-15T17:59:03.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.705+0000 7f16ac1f6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a40af570 con 0x7f16a40b2210 2024-08-15T17:59:03.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16aa793700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f16a40b5040 0x7f16a40b54c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41082/0 (socket says 172.21.15.38:41082) 2024-08-15T17:59:03.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16aa793700 1 -- 172.21.15.38:0/579252780 learned_addr learned my addr 172.21.15.38:0/579252780 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:03.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a9791700 1 --2- 172.21.15.38:0/579252780 >> v2:172.21.15.104:3300/0 conn(0x7f16a40b2210 0x7f16a40b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047820827 0 0) 0x7f16a4159ce0 con 0x7f16a40b5040 2024-08-15T17:59:03.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1690003570 con 0x7f16a40b5040 2024-08-15T17:59:03.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 39661856 0 0) 0x7f1690003570 con 0x7f16a40b5040 2024-08-15T17:59:03.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 >> v2:172.21.15.104:3300/0 conn(0x7f16a40b2210 msgr2=0x7f16a40b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:03.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 --2- 172.21.15.38:0/579252780 >> v2:172.21.15.104:3300/0 conn(0x7f16a40b2210 0x7f16a40b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 >> v1:172.21.15.104:6789/0 conn(0x7f16a40cc840 legacy=0x7f16a40ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a415a2a0 con 0x7f16a40b5040 2024-08-15T17:59:03.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f169c0017f0 con 0x7f16a40b5040 2024-08-15T17:59:03.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 >> v1:172.21.15.38:6789/0 conn(0x7f16a40b5040 legacy=0x7f16a40b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 --2- 172.21.15.38:0/579252780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1690003a60 0x7f1690003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 --2- 172.21.15.38:0/579252780 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1690004810 0x7f1690004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 --2- 172.21.15.38:0/579252780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16900051c0 0x7f169000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40af570 con 0x7f1690003a60 2024-08-15T17:59:03.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169000c0d0 con 0x7f16900051c0 2024-08-15T17:59:03.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169000c290 con 0x7f1690004810 2024-08-15T17:59:03.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a9f92700 1 --2- 172.21.15.38:0/579252780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1690003a60 0x7f1690003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a9791700 1 --2- 172.21.15.38:0/579252780 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1690004810 0x7f1690004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16aa793700 1 --2- 172.21.15.38:0/579252780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16900051c0 0x7f169000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a9f92700 1 -- 172.21.15.38:0/579252780 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1690004810 msgr2=0x7f1690004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:03.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a9f92700 1 --2- 172.21.15.38:0/579252780 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1690004810 0x7f1690004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a9f92700 1 -- 172.21.15.38:0/579252780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16900051c0 msgr2=0x7f169000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:03.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a9f92700 1 --2- 172.21.15.38:0/579252780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16900051c0 0x7f169000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16aa793700 1 -- 172.21.15.38:0/579252780 reap_dead start 2024-08-15T17:59:03.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a9f92700 1 -- 172.21.15.38:0/579252780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690003570 con 0x7f1690003a60 2024-08-15T17:59:03.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a9f92700 1 --2- 172.21.15.38:0/579252780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1690003a60 0x7f1690003e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f16940018b0 tx=0x7f1694007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69b26050d504488 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:03.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694012420 con 0x7f1690003a60 2024-08-15T17:59:03.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1694012580 con 0x7f1690003a60 2024-08-15T17:59:03.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16a8f90700 1 -- 172.21.15.38:0/579252780 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694012920 con 0x7f1690003a60 2024-08-15T17:59:03.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16ac1f6700 1 -- 172.21.15.38:0/579252780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1690003a60 msgr2=0x7f1690003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16ac1f6700 1 --2- 172.21.15.38:0/579252780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1690003a60 0x7f1690003e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f16940018b0 tx=0x7f1694007d30 comp rx=0 tx=0).stop 2024-08-15T17:59:03.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16ac1f6700 1 -- 172.21.15.38:0/579252780 shutdown_connections 2024-08-15T17:59:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16ac1f6700 1 --2- 172.21.15.38:0/579252780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1690003a60 0x7f1690003e70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16ac1f6700 1 -- 172.21.15.38:0/579252780 >> 172.21.15.38:0/579252780 conn(0x7f16a40682c0 msgr2=0x7f16a40686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16ac1f6700 1 -- 172.21.15.38:0/579252780 shutdown_connections 2024-08-15T17:59:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.709+0000 7f16ac1f6700 1 -- 172.21.15.38:0/579252780 wait complete. 2024-08-15T17:59:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16ac1f6700 1 Processor -- start 2024-08-15T17:59:03.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16ac1f6700 1 -- start start 2024-08-15T17:59:03.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16ac1f6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1690003a60 0x7f16a4152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16ac1f6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16a40b3000 0x7f16a40b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16ac1f6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a41542c0 0x7f16a4154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16ac1f6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a41555f0 con 0x7f16a40b3000 2024-08-15T17:59:03.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16ac1f6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40b39b0 con 0x7f1690003a60 2024-08-15T17:59:03.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16ac1f6700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40c5600 con 0x7f16a41542c0 2024-08-15T17:59:03.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16a9791700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16a40b3000 0x7f16a40b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16a9791700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16a40b3000 0x7f16a40b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38826/0 (socket says 172.21.15.38:38826) 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16a9791700 1 -- 172.21.15.38:0/1458634362 learned_addr learned my addr 172.21.15.38:0/1458634362 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16a9f92700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1690003a60 0x7f16a4152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16aa793700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a41542c0 0x7f16a4154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16a9791700 1 -- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a41542c0 msgr2=0x7f16a4154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16a9791700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a41542c0 0x7f16a4154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16a9791700 1 -- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1690003a60 msgr2=0x7f16a4152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16a9791700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1690003a60 0x7f16a4152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.713+0000 7f16a9791700 1 -- 172.21.15.38:0/1458634362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a4155030 con 0x7f16a40b3000 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.717+0000 7f16a9791700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16a40b3000 0x7f16a40b3470 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16a0011d70 tx=0x7f16a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.717+0000 7f1699ffb700 1 -- 172.21.15.38:0/1458634362 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a000fac0 con 0x7f16a40b3000 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.717+0000 7f1699ffb700 1 -- 172.21.15.38:0/1458634362 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a000fc20 con 0x7f16a40b3000 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.717+0000 7f1699ffb700 1 -- 172.21.15.38:0/1458634362 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0014440 con 0x7f16a40b3000 2024-08-15T17:59:03.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.717+0000 7f16ac1f6700 1 -- 172.21.15.38:0/1458634362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a4155310 con 0x7f16a40b3000 2024-08-15T17:59:03.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.717+0000 7f16ac1f6700 1 -- 172.21.15.38:0/1458634362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a4154ca0 con 0x7f16a40b3000 2024-08-15T17:59:03.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.717+0000 7f16ac1f6700 1 -- 172.21.15.38:0/1458634362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a40c50f0 con 0x7f16a40b3000 2024-08-15T17:59:03.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.717+0000 7f1699ffb700 1 -- 172.21.15.38:0/1458634362 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f16a0027480 con 0x7f16a40b3000 2024-08-15T17:59:03.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.721+0000 7f1699ffb700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1688060860 0x7f1688062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:03.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.721+0000 7f1699ffb700 1 -- 172.21.15.38:0/1458634362 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f16a0018030 con 0x7f16a40b3000 2024-08-15T17:59:03.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.721+0000 7f16a9f92700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1688060860 0x7f1688062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:03.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.721+0000 7f16a9f92700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1688060860 0x7f1688062d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1694001010 tx=0x7f1694015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:03.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.725+0000 7f1699ffb700 1 -- 172.21.15.38:0/1458634362 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f16a004dcc0 con 0x7f16a40b3000 2024-08-15T17:59:03.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 -- 172.21.15.38:0/1458634362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f16a40c50f0 con 0x7f16a40b3000 2024-08-15T17:59:03.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f1699ffb700 1 -- 172.21.15.38:0/1458634362 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f16a40c50f0 con 0x7f16a40b3000 2024-08-15T17:59:03.892 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 -- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1688060860 msgr2=0x7f1688062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1688060860 0x7f1688062d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1694001010 tx=0x7f1694015000 comp rx=0 tx=0).stop 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 -- 172.21.15.38:0/1458634362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16a40b3000 msgr2=0x7f16a40b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16a40b3000 0x7f16a40b3470 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16a0011d70 tx=0x7f16a00091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 -- 172.21.15.38:0/1458634362 shutdown_connections 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1688060860 0x7f1688062d10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a41542c0 0x7f16a4154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16a40b3000 0x7f16a40b3470 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 --2- 172.21.15.38:0/1458634362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1690003a60 0x7f16a4152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 -- 172.21.15.38:0/1458634362 >> 172.21.15.38:0/1458634362 conn(0x7f16a40682c0 msgr2=0x7f16a40cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.889+0000 7f16ac1f6700 1 -- 172.21.15.38:0/1458634362 shutdown_connections 2024-08-15T17:59:03.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:03.893+0000 7f16ac1f6700 1 -- 172.21.15.38:0/1458634362 wait complete. 2024-08-15T17:59:03.907 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:59:04.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.017+0000 7f6fa6eca700 1 Processor -- start 2024-08-15T17:59:04.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.017+0000 7f6fa6eca700 1 -- start start 2024-08-15T17:59:04.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa6eca700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6fa0067400 0x7f6fa01593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa6eca700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa00cf670 con 0x7f6fa00cfb00 2024-08-15T17:59:04.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa6eca700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa00cea60 con 0x7f6fa0067030 2024-08-15T17:59:04.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa6eca700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa0159910 con 0x7f6fa0067400 2024-08-15T17:59:04.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa66c9700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6fa0067400 0x7f6fa01593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa56c7700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6fa00cfb00 0x7f6fa0158cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41092/0 (socket says 172.21.15.38:41092) 2024-08-15T17:59:04.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa66c9700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6fa0067400 0x7f6fa01593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38842/0 (socket says 172.21.15.38:38842) 2024-08-15T17:59:04.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa56c7700 1 -- 172.21.15.38:0/2648036884 learned_addr learned my addr 172.21.15.38:0/2648036884 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:04.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa66c9700 1 -- 172.21.15.38:0/2648036884 >> v1:172.21.15.104:6789/0 conn(0x7f6fa0067030 legacy=0x7f6fa00cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa66c9700 1 -- 172.21.15.38:0/2648036884 >> v1:172.21.15.38:6789/0 conn(0x7f6fa00cfb00 legacy=0x7f6fa0158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa66c9700 1 -- 172.21.15.38:0/2648036884 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa00ced40 con 0x7f6fa0067400 2024-08-15T17:59:04.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa66c9700 1 --2- 172.21.15.38:0/2648036884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0067400 0x7f6fa01593d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009e60 tx=0x7f6f98009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33c5f269426e942 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:04.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa4ec6700 1 -- 172.21.15.38:0/2648036884 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9800e070 con 0x7f6fa0067400 2024-08-15T17:59:04.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa4ec6700 1 -- 172.21.15.38:0/2648036884 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9800bc30 con 0x7f6fa0067400 2024-08-15T17:59:04.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa4ec6700 1 -- 172.21.15.38:0/2648036884 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98012430 con 0x7f6fa0067400 2024-08-15T17:59:04.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa6eca700 1 -- 172.21.15.38:0/2648036884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0067400 msgr2=0x7f6fa01593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa6eca700 1 --2- 172.21.15.38:0/2648036884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0067400 0x7f6fa01593d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009e60 tx=0x7f6f98009780 comp rx=0 tx=0).stop 2024-08-15T17:59:04.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa6eca700 1 -- 172.21.15.38:0/2648036884 shutdown_connections 2024-08-15T17:59:04.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa6eca700 1 --2- 172.21.15.38:0/2648036884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0067400 0x7f6fa01593d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa6eca700 1 -- 172.21.15.38:0/2648036884 >> 172.21.15.38:0/2648036884 conn(0x7f6fa00d0b50 msgr2=0x7f6fa00ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:04.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa6eca700 1 -- 172.21.15.38:0/2648036884 shutdown_connections 2024-08-15T17:59:04.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.021+0000 7f6fa6eca700 1 -- 172.21.15.38:0/2648036884 wait complete. 2024-08-15T17:59:04.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa6eca700 1 Processor -- start 2024-08-15T17:59:04.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa6eca700 1 -- start start 2024-08-15T17:59:04.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa6eca700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0067030 0x7f6fa014cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa6eca700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fa0067400 0x7f6fa00b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa6eca700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6fa00cfb00 0x7f6fa015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa6eca700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa015d670 con 0x7f6fa0067030 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa6eca700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa015d060 con 0x7f6fa0067400 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa6eca700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa00b3c50 con 0x7f6fa00cfb00 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa5ec8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0067030 0x7f6fa014cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa5ec8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0067030 0x7f6fa014cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38844/0 (socket says 172.21.15.38:38844) 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa5ec8700 1 -- 172.21.15.38:0/827897674 learned_addr learned my addr 172.21.15.38:0/827897674 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa56c7700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fa0067400 0x7f6fa00b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.025+0000 7f6fa66c9700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6fa00cfb00 0x7f6fa015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6fa5ec8700 1 -- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6fa00cfb00 msgr2=0x7f6fa015bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6fa5ec8700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6fa00cfb00 0x7f6fa015bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6fa5ec8700 1 -- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fa0067400 msgr2=0x7f6fa00b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6fa5ec8700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fa0067400 0x7f6fa00b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6fa5ec8700 1 -- 172.21.15.38:0/827897674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f98009a20 con 0x7f6fa0067030 2024-08-15T17:59:04.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6fa5ec8700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0067030 0x7f6fa014cc00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000c590 tx=0x7f6f9000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:04.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6f9dffb700 1 -- 172.21.15.38:0/827897674 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90011590 con 0x7f6fa0067030 2024-08-15T17:59:04.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6f9dffb700 1 -- 172.21.15.38:0/827897674 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f90011710 con 0x7f6fa0067030 2024-08-15T17:59:04.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6f9dffb700 1 -- 172.21.15.38:0/827897674 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90011ad0 con 0x7f6fa0067030 2024-08-15T17:59:04.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6fa6eca700 1 -- 172.21.15.38:0/827897674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa00b3970 con 0x7f6fa0067030 2024-08-15T17:59:04.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6fa6eca700 1 -- 172.21.15.38:0/827897674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa014d140 con 0x7f6fa0067030 2024-08-15T17:59:04.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6fa6eca700 1 -- 172.21.15.38:0/827897674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa015cec0 con 0x7f6fa0067030 2024-08-15T17:59:04.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.029+0000 7f6f9dffb700 1 -- 172.21.15.38:0/827897674 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6f9001b3b0 con 0x7f6fa0067030 2024-08-15T17:59:04.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.033+0000 7f6f9dffb700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f8c060860 0x7f6f8c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.033+0000 7f6f9dffb700 1 -- 172.21.15.38:0/827897674 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f9007c0b0 con 0x7f6fa0067030 2024-08-15T17:59:04.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.033+0000 7f6fa56c7700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f8c060860 0x7f6f8c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.033+0000 7f6fa56c7700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f8c060860 0x7f6f8c062d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6fa015fc70 tx=0x7f6f94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:04.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.041+0000 7f6f9dffb700 1 -- 172.21.15.38:0/827897674 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6fa015cec0 con 0x7f6fa0067030 2024-08-15T17:59:04.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.205+0000 7f6fa6eca700 1 -- 172.21.15.38:0/827897674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6fa015cec0 con 0x7f6fa0067030 2024-08-15T17:59:04.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6f9dffb700 1 -- 172.21.15.38:0/827897674 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6fa015cec0 con 0x7f6fa0067030 2024-08-15T17:59:04.211 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-08-15T17:59:04.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 -- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f8c060860 msgr2=0x7f6f8c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f8c060860 0x7f6f8c062d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6fa015fc70 tx=0x7f6f94009040 comp rx=0 tx=0).stop 2024-08-15T17:59:04.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 -- 172.21.15.38:0/827897674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0067030 msgr2=0x7f6fa014cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0067030 0x7f6fa014cc00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000c590 tx=0x7f6f9000c8a0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 -- 172.21.15.38:0/827897674 shutdown_connections 2024-08-15T17:59:04.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f8c060860 0x7f6f8c062d10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6fa00cfb00 0x7f6fa015bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fa0067400 0x7f6fa00b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 --2- 172.21.15.38:0/827897674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0067030 0x7f6fa014cc00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 -- 172.21.15.38:0/827897674 >> 172.21.15.38:0/827897674 conn(0x7f6fa00d0b50 msgr2=0x7f6fa00c6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:04.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 -- 172.21.15.38:0/827897674 shutdown_connections 2024-08-15T17:59:04.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.209+0000 7f6fa6eca700 1 -- 172.21.15.38:0/827897674 wait complete. 2024-08-15T17:59:04.288 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2024-08-15T17:59:04.288 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T17:59:04.289 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:59:04.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.397+0000 7f2c0ffff700 1 Processor -- start 2024-08-15T17:59:04.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.397+0000 7f2c0ffff700 1 -- start start 2024-08-15T17:59:04.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2c10157050 0x7f2c100b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0ffff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c100af5c0 con 0x7f2c10156c80 2024-08-15T17:59:04.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0effd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2c10156c80 0x7f2c100b1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41100/0 (socket says 172.21.15.38:41100) 2024-08-15T17:59:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0ffff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c100ae9f0 con 0x7f2c100b0dc0 2024-08-15T17:59:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0effd700 1 -- 172.21.15.38:0/3975766131 learned_addr learned my addr 172.21.15.38:0/3975766131 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0ffff700 1 -- 172.21.15.38:0/3975766131 --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c101574b0 con 0x7f2c10157050 2024-08-15T17:59:04.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0f7fe700 1 --2- 172.21.15.38:0/3975766131 >> v2:172.21.15.104:3300/0 conn(0x7f2c10157050 0x7f2c100b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702813742 0 0) 0x7f2c100af5c0 con 0x7f2c10156c80 2024-08-15T17:59:04.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bfc003570 con 0x7f2c10156c80 2024-08-15T17:59:04.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1924973881 0 0) 0x7f2bfc003570 con 0x7f2c10156c80 2024-08-15T17:59:04.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 >> v2:172.21.15.104:3300/0 conn(0x7f2c10157050 msgr2=0x7f2c100b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:04.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 --2- 172.21.15.38:0/3975766131 >> v2:172.21.15.104:3300/0 conn(0x7f2c10157050 0x7f2c100b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 >> v1:172.21.15.104:6789/0 conn(0x7f2c100b0dc0 legacy=0x7f2c100afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c100b2320 con 0x7f2c10156c80 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f2c00001470 con 0x7f2c10156c80 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 >> v1:172.21.15.38:6789/0 conn(0x7f2c10156c80 legacy=0x7f2c100b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bfc003a60 0x7f2bfc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2bfc004810 0x7f2bfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bfc0051c0 0x7f2bfc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c101574b0 con 0x7f2bfc0051c0 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc00c0b0 con 0x7f2bfc003a60 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc00c270 con 0x7f2bfc004810 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0f7fe700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bfc0051c0 0x7f2bfc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0effd700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bfc003a60 0x7f2bfc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c07fff700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2bfc004810 0x7f2bfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0f7fe700 1 -- 172.21.15.38:0/3975766131 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2bfc004810 msgr2=0x7f2bfc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:04.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0f7fe700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2bfc004810 0x7f2bfc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0f7fe700 1 -- 172.21.15.38:0/3975766131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bfc003a60 msgr2=0x7f2bfc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:04.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0f7fe700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bfc003a60 0x7f2bfc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c0f7fe700 1 -- 172.21.15.38:0/3975766131 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc003570 con 0x7f2bfc0051c0 2024-08-15T17:59:04.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.401+0000 7f2c07fff700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2bfc004810 0x7f2bfc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:04.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0f7fe700 1 -- 172.21.15.38:0/3975766131 reap_dead start 2024-08-15T17:59:04.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0f7fe700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bfc0051c0 0x7f2bfc00ba60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800e210 tx=0x7f2c0800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff2092276a4a413 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:04.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800eac0 con 0x7f2bfc0051c0 2024-08-15T17:59:04.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0800f040 con 0x7f2bfc0051c0 2024-08-15T17:59:04.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0e7fc700 1 -- 172.21.15.38:0/3975766131 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800ee70 con 0x7f2bfc0051c0 2024-08-15T17:59:04.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 -- 172.21.15.38:0/3975766131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bfc0051c0 msgr2=0x7f2bfc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bfc0051c0 0x7f2bfc00ba60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800e210 tx=0x7f2c0800b040 comp rx=0 tx=0).stop 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 -- 172.21.15.38:0/3975766131 shutdown_connections 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 --2- 172.21.15.38:0/3975766131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bfc0051c0 0x7f2bfc00ba60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 -- 172.21.15.38:0/3975766131 >> 172.21.15.38:0/3975766131 conn(0x7f2c100ce990 msgr2=0x7f2c1006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 -- 172.21.15.38:0/3975766131 shutdown_connections 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 -- 172.21.15.38:0/3975766131 wait complete. 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 Processor -- start 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 -- start start 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bfc0051c0 0x7f2c1015a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c1015bea0 0x7f2c1014c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c1014c6d0 0x7f2c101575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1014cb40 con 0x7f2c1015bea0 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1015c310 con 0x7f2bfc0051c0 2024-08-15T17:59:04.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0ffff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1015ae00 con 0x7f2c1014c6d0 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c07fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c1015bea0 0x7f2c1014c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c07fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c1015bea0 0x7f2c1014c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38870/0 (socket says 172.21.15.38:38870) 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c07fff700 1 -- 172.21.15.38:0/4026843317 learned_addr learned my addr 172.21.15.38:0/4026843317 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0f7fe700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c1014c6d0 0x7f2c101575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.405+0000 7f2c0effd700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bfc0051c0 0x7f2c1015a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c07fff700 1 -- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c1014c6d0 msgr2=0x7f2c101575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c07fff700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c1014c6d0 0x7f2c101575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c07fff700 1 -- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bfc0051c0 msgr2=0x7f2c1015a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c07fff700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bfc0051c0 0x7f2c1015a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c07fff700 1 -- 172.21.15.38:0/4026843317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1015b0e0 con 0x7f2c1015bea0 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c0effd700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bfc0051c0 0x7f2c1015a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c0f7fe700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c1014c6d0 0x7f2c101575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c07fff700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c1015bea0 0x7f2c1014c190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8001a50 tx=0x7f2bf8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:04.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c06ffd700 1 -- 172.21.15.38:0/4026843317 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8016030 con 0x7f2c1015bea0 2024-08-15T17:59:04.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c06ffd700 1 -- 172.21.15.38:0/4026843317 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf8010070 con 0x7f2c1015bea0 2024-08-15T17:59:04.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c0ffff700 1 -- 172.21.15.38:0/4026843317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1015b3c0 con 0x7f2c1015bea0 2024-08-15T17:59:04.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c06ffd700 1 -- 172.21.15.38:0/4026843317 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf800a750 con 0x7f2c1015bea0 2024-08-15T17:59:04.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c0ffff700 1 -- 172.21.15.38:0/4026843317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1015b980 con 0x7f2c1015bea0 2024-08-15T17:59:04.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c0ffff700 1 -- 172.21.15.38:0/4026843317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c10158300 con 0x7f2c1015bea0 2024-08-15T17:59:04.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c06ffd700 1 -- 172.21.15.38:0/4026843317 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2bf8019070 con 0x7f2c1015bea0 2024-08-15T17:59:04.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.409+0000 7f2c06ffd700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2bf0060860 0x7f2bf0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.413+0000 7f2c06ffd700 1 -- 172.21.15.38:0/4026843317 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2bf8017070 con 0x7f2c1015bea0 2024-08-15T17:59:04.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.413+0000 7f2c0effd700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2bf0060860 0x7f2bf0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.413+0000 7f2c0effd700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2bf0060860 0x7f2bf0062d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2c00000e10 tx=0x7f2c00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:04.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.417+0000 7f2c06ffd700 1 -- 172.21.15.38:0/4026843317 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2c10158300 con 0x7f2c1015bea0 2024-08-15T17:59:04.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.585+0000 7f2c0ffff700 1 -- 172.21.15.38:0/4026843317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c10158300 con 0x7f2c1015bea0 2024-08-15T17:59:04.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c06ffd700 1 -- 172.21.15.38:0/4026843317 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f2c10158300 con 0x7f2c1015bea0 2024-08-15T17:59:04.592 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:04.592 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:59:04.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 -- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2bf0060860 msgr2=0x7f2bf0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2bf0060860 0x7f2bf0062d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2c00000e10 tx=0x7f2c00009040 comp rx=0 tx=0).stop 2024-08-15T17:59:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 -- 172.21.15.38:0/4026843317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c1015bea0 msgr2=0x7f2c1014c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c1015bea0 0x7f2c1014c190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8001a50 tx=0x7f2bf8009040 comp rx=0 tx=0).stop 2024-08-15T17:59:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 -- 172.21.15.38:0/4026843317 shutdown_connections 2024-08-15T17:59:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2bf0060860 0x7f2bf0062d10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c1014c6d0 0x7f2c101575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c1015bea0 0x7f2c1014c190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 --2- 172.21.15.38:0/4026843317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bfc0051c0 0x7f2c1015a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 -- 172.21.15.38:0/4026843317 >> 172.21.15.38:0/4026843317 conn(0x7f2c100ce990 msgr2=0x7f2c1006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 -- 172.21.15.38:0/4026843317 shutdown_connections 2024-08-15T17:59:04.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.589+0000 7f2c0ffff700 1 -- 172.21.15.38:0/4026843317 wait complete. 2024-08-15T17:59:04.606 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:58:34.079267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:58:34.621969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:59:04.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:59:04.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cf4e5700 1 Processor -- start 2024-08-15T17:59:04.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cf4e5700 1 -- start start 2024-08-15T17:59:04.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cf4e5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc9c80cbab0 0x7fc9c80cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cf4e5700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9c80ad6e0 0x7fc9c80adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cf4e5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c80cc450 con 0x7fc9c8140910 2024-08-15T17:59:04.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cf4e5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c80c8c30 con 0x7fc9c80cbab0 2024-08-15T17:59:04.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cf4e5700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c80c3e30 con 0x7fc9c80ad6e0 2024-08-15T17:59:04.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cdce2700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc9c80cbab0 0x7fc9c80cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cdce2700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc9c80cbab0 0x7fc9c80cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38876/0 (socket says 172.21.15.38:38876) 2024-08-15T17:59:04.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cdce2700 1 -- 172.21.15.38:0/4021897577 learned_addr learned my addr 172.21.15.38:0/4021897577 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:04.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9ce4e3700 1 --1- 172.21.15.38:0/4021897577 >> v1:172.21.15.38:6789/0 conn(0x7fc9c8140910 0x7fc9c80c8520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41106/0 (socket says 172.21.15.38:41106) 2024-08-15T17:59:04.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cece4700 1 --2- 172.21.15.38:0/4021897577 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9c80ad6e0 0x7fc9c80adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cdce2700 1 -- 172.21.15.38:0/4021897577 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9c80ad6e0 msgr2=0x7fc9c80adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:04.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.717+0000 7fc9cdce2700 1 --2- 172.21.15.38:0/4021897577 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9c80ad6e0 0x7fc9c80adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cdce2700 1 -- 172.21.15.38:0/4021897577 >> v1:172.21.15.38:6789/0 conn(0x7fc9c8140910 legacy=0x7fc9c80c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cdce2700 1 -- 172.21.15.38:0/4021897577 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c80c4110 con 0x7fc9c80cbab0 2024-08-15T17:59:04.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cdce2700 1 --2- 172.21.15.38:0/4021897577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9c80cbab0 0x7fc9c80cbf10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40099b0 tx=0x7fc9c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9315b0ee1dd15136 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:04.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cd4e1700 1 -- 172.21.15.38:0/4021897577 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4009ea0 con 0x7fc9c80cbab0 2024-08-15T17:59:04.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cd4e1700 1 -- 172.21.15.38:0/4021897577 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c400b860 con 0x7fc9c80cbab0 2024-08-15T17:59:04.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cd4e1700 1 -- 172.21.15.38:0/4021897577 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c400bc00 con 0x7fc9c80cbab0 2024-08-15T17:59:04.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/4021897577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9c80cbab0 msgr2=0x7fc9c80cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cf4e5700 1 --2- 172.21.15.38:0/4021897577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9c80cbab0 0x7fc9c80cbf10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40099b0 tx=0x7fc9c4009250 comp rx=0 tx=0).stop 2024-08-15T17:59:04.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/4021897577 shutdown_connections 2024-08-15T17:59:04.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cf4e5700 1 --2- 172.21.15.38:0/4021897577 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9c80ad6e0 0x7fc9c80adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cf4e5700 1 --2- 172.21.15.38:0/4021897577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9c80cbab0 0x7fc9c80cbf10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/4021897577 >> 172.21.15.38:0/4021897577 conn(0x7fc9c80cdee0 msgr2=0x7fc9c80ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:04.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/4021897577 shutdown_connections 2024-08-15T17:59:04.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.721+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/4021897577 wait complete. 2024-08-15T17:59:04.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cf4e5700 1 Processor -- start 2024-08-15T17:59:04.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cf4e5700 1 -- start start 2024-08-15T17:59:04.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cf4e5700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9c80ad6e0 0x7fc9c812c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cf4e5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9c80cbab0 0x7fc9c81576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cf4e5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9c8140910 0x7fc9c815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cf4e5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c813ff50 con 0x7fc9c8140910 2024-08-15T17:59:04.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cf4e5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c81580f0 con 0x7fc9c80cbab0 2024-08-15T17:59:04.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cf4e5700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c81583d0 con 0x7fc9c80ad6e0 2024-08-15T17:59:04.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cece4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9c8140910 0x7fc9c815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cece4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9c8140910 0x7fc9c815a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38878/0 (socket says 172.21.15.38:38878) 2024-08-15T17:59:04.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cece4700 1 -- 172.21.15.38:0/2513476338 learned_addr learned my addr 172.21.15.38:0/2513476338 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:04.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9ce4e3700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9c80ad6e0 0x7fc9c812c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cdce2700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9c80cbab0 0x7fc9c81576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cece4700 1 -- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9c80ad6e0 msgr2=0x7fc9c812c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cece4700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9c80ad6e0 0x7fc9c812c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cece4700 1 -- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9c80cbab0 msgr2=0x7fc9c81576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cece4700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9c80cbab0 0x7fc9c81576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cece4700 1 -- 172.21.15.38:0/2513476338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c40094f0 con 0x7fc9c8140910 2024-08-15T17:59:04.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9ce4e3700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9c80ad6e0 0x7fc9c812c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:04.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9cece4700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9c8140910 0x7fc9c815a3b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009df0 tx=0x7fc9c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:04.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.725+0000 7fc9ba7fc700 1 -- 172.21.15.38:0/2513476338 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000cb50 con 0x7fc9c8140910 2024-08-15T17:59:04.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.729+0000 7fc9ba7fc700 1 -- 172.21.15.38:0/2513476338 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c000f040 con 0x7fc9c8140910 2024-08-15T17:59:04.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.729+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/2513476338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c81586b0 con 0x7fc9c8140910 2024-08-15T17:59:04.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.729+0000 7fc9ba7fc700 1 -- 172.21.15.38:0/2513476338 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0014480 con 0x7fc9c8140910 2024-08-15T17:59:04.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.729+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/2513476338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c8158c70 con 0x7fc9c8140910 2024-08-15T17:59:04.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.729+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/2513476338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c80638f0 con 0x7fc9c8140910 2024-08-15T17:59:04.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.729+0000 7fc9ba7fc700 1 -- 172.21.15.38:0/2513476338 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc9c0014830 con 0x7fc9c8140910 2024-08-15T17:59:04.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.733+0000 7fc9ba7fc700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9ac060860 0x7fc9ac062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:04.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.733+0000 7fc9ba7fc700 1 -- 172.21.15.38:0/2513476338 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc9c007d460 con 0x7fc9c8140910 2024-08-15T17:59:04.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.733+0000 7fc9ce4e3700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9ac060860 0x7fc9ac062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:04.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.733+0000 7fc9ce4e3700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9ac060860 0x7fc9ac062d10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc007120 tx=0x7fc9bc007d80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:04.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.737+0000 7fc9ba7fc700 1 -- 172.21.15.38:0/2513476338 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc9c004d8b0 con 0x7fc9c8140910 2024-08-15T17:59:04.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.901+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/2513476338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9c80c9940 con 0x7fc9c8140910 2024-08-15T17:59:04.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.901+0000 7fc9ba7fc700 1 -- 172.21.15.38:0/2513476338 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9c0050ed0 con 0x7fc9c8140910 2024-08-15T17:59:04.904 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T17:59:04.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.901+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9ac060860 msgr2=0x7fc9ac062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.901+0000 7fc9cf4e5700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9ac060860 0x7fc9ac062d10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc007120 tx=0x7fc9bc007d80 comp rx=0 tx=0).stop 2024-08-15T17:59:04.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.905+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/2513476338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9c8140910 msgr2=0x7fc9c815a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:04.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.905+0000 7fc9cf4e5700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9c8140910 0x7fc9c815a3b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009df0 tx=0x7fc9c000b040 comp rx=0 tx=0).stop 2024-08-15T17:59:04.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.905+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/2513476338 shutdown_connections 2024-08-15T17:59:04.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.905+0000 7fc9cf4e5700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc9ac060860 0x7fc9ac062d10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.905+0000 7fc9cf4e5700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9c8140910 0x7fc9c815a3b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.905+0000 7fc9cf4e5700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9c80cbab0 0x7fc9c81576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.905+0000 7fc9cf4e5700 1 --2- 172.21.15.38:0/2513476338 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9c80ad6e0 0x7fc9c812c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:04.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.905+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/2513476338 >> 172.21.15.38:0/2513476338 conn(0x7fc9c80cdee0 msgr2=0x7fc9c80cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:04.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.905+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/2513476338 shutdown_connections 2024-08-15T17:59:04.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:04.905+0000 7fc9cf4e5700 1 -- 172.21.15.38:0/2513476338 wait complete. 2024-08-15T17:59:04.919 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:59:05.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.033+0000 7f7555a6e700 1 Processor -- start 2024-08-15T17:59:05.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.033+0000 7f7555a6e700 1 -- start start 2024-08-15T17:59:05.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.033+0000 7f7555a6e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f75500cfb00 0x7f7550067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.033+0000 7f7555a6e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7550158cc0 0x7f75501590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.033+0000 7f7555a6e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75500cf670 con 0x7f7550067030 2024-08-15T17:59:05.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.033+0000 7f7555a6e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75500cea60 con 0x7f75500cfb00 2024-08-15T17:59:05.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.033+0000 7f7555a6e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75500ced40 con 0x7f7550158cc0 2024-08-15T17:59:05.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.033+0000 7f7547fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f75500cfb00 0x7f7550067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7547fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f75500cfb00 0x7f7550067400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38888/0 (socket says 172.21.15.38:38888) 2024-08-15T17:59:05.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7547fff700 1 -- 172.21.15.38:0/4285568508 learned_addr learned my addr 172.21.15.38:0/4285568508 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:05.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.033+0000 7f7554a6c700 1 --1- 172.21.15.38:0/4285568508 >> v1:172.21.15.38:6789/0 conn(0x7f7550067030 0x7f75500cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41120/0 (socket says 172.21.15.38:41120) 2024-08-15T17:59:05.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f755526d700 1 --2- 172.21.15.38:0/4285568508 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7550158cc0 0x7f75501590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7547fff700 1 -- 172.21.15.38:0/4285568508 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7550158cc0 msgr2=0x7f75501590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:05.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7547fff700 1 --2- 172.21.15.38:0/4285568508 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7550158cc0 0x7f75501590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7547fff700 1 -- 172.21.15.38:0/4285568508 >> v1:172.21.15.38:6789/0 conn(0x7f7550067030 legacy=0x7f75500cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7547fff700 1 -- 172.21.15.38:0/4285568508 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500cf020 con 0x7f75500cfb00 2024-08-15T17:59:05.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7547fff700 1 --2- 172.21.15.38:0/4285568508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75500cfb00 0x7f7550067400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f75400099b0 tx=0x7f7540009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c58c9ee20ffc92c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:05.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f75477fe700 1 -- 172.21.15.38:0/4285568508 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540009ea0 con 0x7f75500cfb00 2024-08-15T17:59:05.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f75477fe700 1 -- 172.21.15.38:0/4285568508 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754000b860 con 0x7f75500cfb00 2024-08-15T17:59:05.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f75477fe700 1 -- 172.21.15.38:0/4285568508 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754000bc00 con 0x7f75500cfb00 2024-08-15T17:59:05.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7555a6e700 1 -- 172.21.15.38:0/4285568508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75500cfb00 msgr2=0x7f7550067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7555a6e700 1 --2- 172.21.15.38:0/4285568508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75500cfb00 0x7f7550067400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f75400099b0 tx=0x7f7540009250 comp rx=0 tx=0).stop 2024-08-15T17:59:05.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7555a6e700 1 -- 172.21.15.38:0/4285568508 shutdown_connections 2024-08-15T17:59:05.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7555a6e700 1 --2- 172.21.15.38:0/4285568508 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7550158cc0 0x7f75501590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7555a6e700 1 --2- 172.21.15.38:0/4285568508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75500cfb00 0x7f7550067400 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7555a6e700 1 -- 172.21.15.38:0/4285568508 >> 172.21.15.38:0/4285568508 conn(0x7f75500d0b50 msgr2=0x7f75500ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:05.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7555a6e700 1 -- 172.21.15.38:0/4285568508 shutdown_connections 2024-08-15T17:59:05.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.037+0000 7f7555a6e700 1 -- 172.21.15.38:0/4285568508 wait complete. 2024-08-15T17:59:05.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.041+0000 7f7555a6e700 1 Processor -- start 2024-08-15T17:59:05.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.041+0000 7f7555a6e700 1 -- start start 2024-08-15T17:59:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.041+0000 7f7555a6e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7550067030 0x7f7550143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.041+0000 7f7555a6e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75500cfb00 0x7f75500b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.041+0000 7f7555a6e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7550158cc0 0x7f75500b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.041+0000 7f7555a6e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755015a790 con 0x7f7550067030 2024-08-15T17:59:05.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.041+0000 7f7555a6e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7550159b80 con 0x7f75500cfb00 2024-08-15T17:59:05.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.041+0000 7f7555a6e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7550159e60 con 0x7f7550158cc0 2024-08-15T17:59:05.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.041+0000 7f7554a6c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7550067030 0x7f7550143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.041+0000 7f7554a6c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7550067030 0x7f7550143a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38900/0 (socket says 172.21.15.38:38900) 2024-08-15T17:59:05.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.041+0000 7f7554a6c700 1 -- 172.21.15.38:0/2023247432 learned_addr learned my addr 172.21.15.38:0/2023247432 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:05.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f755526d700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7550158cc0 0x7f75500b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7547fff700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75500cfb00 0x7f75500b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7554a6c700 1 -- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7550158cc0 msgr2=0x7f75500b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7554a6c700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7550158cc0 0x7f75500b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7554a6c700 1 -- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75500cfb00 msgr2=0x7f75500b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7554a6c700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75500cfb00 0x7f75500b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7554a6c700 1 -- 172.21.15.38:0/2023247432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400094f0 con 0x7f7550067030 2024-08-15T17:59:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7547fff700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75500cfb00 0x7f75500b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7554a6c700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7550067030 0x7f7550143a90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f754c00c7c0 tx=0x7f754c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7544ff9700 1 -- 172.21.15.38:0/2023247432 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c005a70 con 0x7f7550067030 2024-08-15T17:59:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7544ff9700 1 -- 172.21.15.38:0/2023247432 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c005bd0 con 0x7f7550067030 2024-08-15T17:59:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7555a6e700 1 -- 172.21.15.38:0/2023247432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755015a140 con 0x7f7550067030 2024-08-15T17:59:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7544ff9700 1 -- 172.21.15.38:0/2023247432 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c0070d0 con 0x7f7550067030 2024-08-15T17:59:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7555a6e700 1 -- 172.21.15.38:0/2023247432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7550143fd0 con 0x7f7550067030 2024-08-15T17:59:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7544ff9700 1 -- 172.21.15.38:0/2023247432 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f754c007480 con 0x7f7550067030 2024-08-15T17:59:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.045+0000 7f7555a6e700 1 -- 172.21.15.38:0/2023247432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75500638d0 con 0x7f7550067030 2024-08-15T17:59:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.049+0000 7f7544ff9700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f753c060860 0x7f753c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.049+0000 7f7544ff9700 1 -- 172.21.15.38:0/2023247432 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f754c07c120 con 0x7f7550067030 2024-08-15T17:59:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.049+0000 7f7547fff700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f753c060860 0x7f753c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.049+0000 7f7547fff700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f753c060860 0x7f753c062d10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f75400099b0 tx=0x7f7540017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:05.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.057+0000 7f7544ff9700 1 -- 172.21.15.38:0/2023247432 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f754c04c510 con 0x7f7550067030 2024-08-15T17:59:05.216 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:59:05.219 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:59:05.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.221+0000 7f7555a6e700 1 -- 172.21.15.38:0/2023247432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75500703a0 con 0x7f7550067030 2024-08-15T17:59:05.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.221+0000 7f7544ff9700 1 -- 172.21.15.38:0/2023247432 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f754c04fb30 con 0x7f7550067030 2024-08-15T17:59:05.224 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T17:59:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 -- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f753c060860 msgr2=0x7f753c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f753c060860 0x7f753c062d10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f75400099b0 tx=0x7f7540017000 comp rx=0 tx=0).stop 2024-08-15T17:59:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 -- 172.21.15.38:0/2023247432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7550067030 msgr2=0x7f7550143a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7550067030 0x7f7550143a90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f754c00c7c0 tx=0x7f754c007f70 comp rx=0 tx=0).stop 2024-08-15T17:59:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 -- 172.21.15.38:0/2023247432 shutdown_connections 2024-08-15T17:59:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f753c060860 0x7f753c062d10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7550158cc0 0x7f75500b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75500cfb00 0x7f75500b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 --2- 172.21.15.38:0/2023247432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7550067030 0x7f7550143a90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 -- 172.21.15.38:0/2023247432 >> 172.21.15.38:0/2023247432 conn(0x7f75500d0b50 msgr2=0x7f75500ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:05.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 -- 172.21.15.38:0/2023247432 shutdown_connections 2024-08-15T17:59:05.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.225+0000 7f7555a6e700 1 -- 172.21.15.38:0/2023247432 wait complete. 2024-08-15T17:59:05.238 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:59:05.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.433+0000 7fac0f092700 1 Processor -- start 2024-08-15T17:59:05.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.433+0000 7fac0f092700 1 -- start start 2024-08-15T17:59:05.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.433+0000 7fac0f092700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fac081596d0 0x7fac08066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.433+0000 7fac0f092700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 0x7fac08067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.433+0000 7fac0f092700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac080cba90 con 0x7fac08159300 2024-08-15T17:59:05.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.433+0000 7fac0f092700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac080b2b60 con 0x7fac081596d0 2024-08-15T17:59:05.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.433+0000 7fac0f092700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac080b2940 con 0x7fac0815dbc0 2024-08-15T17:59:05.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac07fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fac081596d0 0x7fac08066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0d62f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 0x7fac08067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac07fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fac081596d0 0x7fac08066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38906/0 (socket says 172.21.15.38:38906) 2024-08-15T17:59:05.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0ce2e700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fac08159300 0x7fac080b2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:55812/0 (socket says 172.21.15.38:55812) 2024-08-15T17:59:05.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac07fff700 1 -- 172.21.15.38:0/4115319450 learned_addr learned my addr 172.21.15.38:0/4115319450 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:05.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac077fe700 1 -- 172.21.15.38:0/4115319450 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512615067 0 0) 0x7fac080cba90 con 0x7fac08159300 2024-08-15T17:59:05.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac077fe700 1 -- 172.21.15.38:0/4115319450 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf4003570 con 0x7fac08159300 2024-08-15T17:59:05.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0d62f700 1 -- 172.21.15.38:0/4115319450 >> v2:172.21.15.38:3300/0 conn(0x7fac081596d0 msgr2=0x7fac08066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:05.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0d62f700 1 --2- 172.21.15.38:0/4115319450 >> v2:172.21.15.38:3300/0 conn(0x7fac081596d0 0x7fac08066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0d62f700 1 -- 172.21.15.38:0/4115319450 >> v1:172.21.15.104:6789/0 conn(0x7fac08159300 legacy=0x7fac080b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0d62f700 1 -- 172.21.15.38:0/4115319450 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac080cb160 con 0x7fac0815dbc0 2024-08-15T17:59:05.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac07fff700 1 --2- 172.21.15.38:0/4115319450 >> v2:172.21.15.38:3300/0 conn(0x7fac081596d0 0x7fac08066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:05.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0d62f700 1 --2- 172.21.15.38:0/4115319450 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 0x7fac08067340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fac00009e60 tx=0x7fac00009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a63db0e1a2cead server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:05.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac077fe700 1 -- 172.21.15.38:0/4115319450 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0000e070 con 0x7fac0815dbc0 2024-08-15T17:59:05.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac077fe700 1 -- 172.21.15.38:0/4115319450 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0000bab0 con 0x7fac0815dbc0 2024-08-15T17:59:05.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac077fe700 1 -- 172.21.15.38:0/4115319450 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0000be70 con 0x7fac0815dbc0 2024-08-15T17:59:05.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0f092700 1 -- 172.21.15.38:0/4115319450 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 msgr2=0x7fac08067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0f092700 1 --2- 172.21.15.38:0/4115319450 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 0x7fac08067340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fac00009e60 tx=0x7fac00009780 comp rx=0 tx=0).stop 2024-08-15T17:59:05.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0f092700 1 -- 172.21.15.38:0/4115319450 shutdown_connections 2024-08-15T17:59:05.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0f092700 1 --2- 172.21.15.38:0/4115319450 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 0x7fac08067340 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0f092700 1 --2- 172.21.15.38:0/4115319450 >> v2:172.21.15.38:3300/0 conn(0x7fac081596d0 0x7fac08066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0f092700 1 -- 172.21.15.38:0/4115319450 >> 172.21.15.38:0/4115319450 conn(0x7fac080682c0 msgr2=0x7fac080686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:05.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0f092700 1 -- 172.21.15.38:0/4115319450 shutdown_connections 2024-08-15T17:59:05.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.437+0000 7fac0f092700 1 -- 172.21.15.38:0/4115319450 wait complete. 2024-08-15T17:59:05.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0f092700 1 Processor -- start 2024-08-15T17:59:05.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0f092700 1 -- start start 2024-08-15T17:59:05.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0f092700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac08159300 0x7fac08151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0f092700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac081596d0 0x7fac080c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0f092700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 0x7fac08152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0f092700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac080c40a0 con 0x7fac081596d0 2024-08-15T17:59:05.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0f092700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac080c4620 con 0x7fac08159300 2024-08-15T17:59:05.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0f092700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac08155d60 con 0x7fac0815dbc0 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac07fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac081596d0 0x7fac080c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac07fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac081596d0 0x7fac080c6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38912/0 (socket says 172.21.15.38:38912) 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac07fff700 1 -- 172.21.15.38:0/4097171932 learned_addr learned my addr 172.21.15.38:0/4097171932 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0d62f700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 0x7fac08152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0ce2e700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac08159300 0x7fac08151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac07fff700 1 -- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 msgr2=0x7fac08152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac07fff700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 0x7fac08152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac07fff700 1 -- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac08159300 msgr2=0x7fac08151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac07fff700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac08159300 0x7fac08151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac07fff700 1 -- 172.21.15.38:0/4097171932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf80094f0 con 0x7fac081596d0 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0d62f700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 0x7fac08152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.441+0000 7fac0ce2e700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac08159300 0x7fac08151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:05.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.445+0000 7fac07fff700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac081596d0 0x7fac080c6510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fabf800ec50 tx=0x7fabf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:05.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.445+0000 7fac04ff9700 1 -- 172.21.15.38:0/4097171932 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf800c9d0 con 0x7fac081596d0 2024-08-15T17:59:05.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.445+0000 7fac04ff9700 1 -- 172.21.15.38:0/4097171932 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf800cb30 con 0x7fac081596d0 2024-08-15T17:59:05.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.445+0000 7fac0f092700 1 -- 172.21.15.38:0/4097171932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac00009a20 con 0x7fac081596d0 2024-08-15T17:59:05.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.445+0000 7fac04ff9700 1 -- 172.21.15.38:0/4097171932 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf8013470 con 0x7fac081596d0 2024-08-15T17:59:05.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.445+0000 7fac0f092700 1 -- 172.21.15.38:0/4097171932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac080c42b0 con 0x7fac081596d0 2024-08-15T17:59:05.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.445+0000 7fac0f092700 1 -- 172.21.15.38:0/4097171932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac080c4790 con 0x7fac081596d0 2024-08-15T17:59:05.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.449+0000 7fac04ff9700 1 -- 172.21.15.38:0/4097171932 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fabf8013860 con 0x7fac081596d0 2024-08-15T17:59:05.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.449+0000 7fac04ff9700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fabe8060860 0x7fabe8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.453+0000 7fac04ff9700 1 -- 172.21.15.38:0/4097171932 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fabf807d840 con 0x7fac081596d0 2024-08-15T17:59:05.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.453+0000 7fac0ce2e700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fabe8060860 0x7fabe8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.453+0000 7fac0ce2e700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fabe8060860 0x7fabe8062d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fac080b4670 tx=0x7fabf0007c80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:05.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.453+0000 7fac04ff9700 1 -- 172.21.15.38:0/4097171932 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fabf804dc70 con 0x7fac081596d0 2024-08-15T17:59:05.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 -- 172.21.15.38:0/4097171932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac080645c0 con 0x7fac081596d0 2024-08-15T17:59:05.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac04ff9700 1 -- 172.21.15.38:0/4097171932 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabf8017020 con 0x7fac081596d0 2024-08-15T17:59:05.624 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-08-15T17:59:05.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 -- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fabe8060860 msgr2=0x7fabe8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fabe8060860 0x7fabe8062d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fac080b4670 tx=0x7fabf0007c80 comp rx=0 tx=0).stop 2024-08-15T17:59:05.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 -- 172.21.15.38:0/4097171932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac081596d0 msgr2=0x7fac080c6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac081596d0 0x7fac080c6510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fabf800ec50 tx=0x7fabf8009250 comp rx=0 tx=0).stop 2024-08-15T17:59:05.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 -- 172.21.15.38:0/4097171932 shutdown_connections 2024-08-15T17:59:05.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fabe8060860 0x7fabe8062d10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac0815dbc0 0x7fac08152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac081596d0 0x7fac080c6510 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 --2- 172.21.15.38:0/4097171932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac08159300 0x7fac08151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 -- 172.21.15.38:0/4097171932 >> 172.21.15.38:0/4097171932 conn(0x7fac080682c0 msgr2=0x7fac08066e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:05.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 -- 172.21.15.38:0/4097171932 shutdown_connections 2024-08-15T17:59:05.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.621+0000 7fac0f092700 1 -- 172.21.15.38:0/4097171932 wait complete. 2024-08-15T17:59:05.638 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-08-15T17:59:05.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.741+0000 7f0dd9ce2700 1 Processor -- start 2024-08-15T17:59:05.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.741+0000 7f0dd9ce2700 1 -- start start 2024-08-15T17:59:05.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.741+0000 7f0dd9ce2700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0dd40aac50 0x7f0dd415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.741+0000 7f0dd9ce2700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0dd40b3b30 0x7f0dd40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.741+0000 7f0dd9ce2700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd40b4470 con 0x7f0dd415a2e0 2024-08-15T17:59:05.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.741+0000 7f0dd9ce2700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd415bc90 con 0x7f0dd40b3b30 2024-08-15T17:59:05.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.741+0000 7f0dd9ce2700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd415b010 con 0x7f0dd40aac50 2024-08-15T17:59:05.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.741+0000 7f0dd2ffd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0dd415a2e0 0x7f0dd41574a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41124/0 (socket says 172.21.15.38:41124) 2024-08-15T17:59:05.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.741+0000 7f0dd2ffd700 1 -- 172.21.15.38:0/2920394880 learned_addr learned my addr 172.21.15.38:0/2920394880 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:05.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.741+0000 7f0dd3fff700 1 --2- 172.21.15.38:0/2920394880 >> v2:172.21.15.38:3300/0 conn(0x7f0dd40b3b30 0x7f0dd40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.741+0000 7f0dd37fe700 1 --2- 172.21.15.38:0/2920394880 >> v2:172.21.15.104:3300/0 conn(0x7f0dd40aac50 0x7f0dd415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd27fc700 1 -- 172.21.15.38:0/2920394880 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761849863 0 0) 0x7f0dd40b4470 con 0x7f0dd415a2e0 2024-08-15T17:59:05.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd27fc700 1 -- 172.21.15.38:0/2920394880 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dc0003570 con 0x7f0dd415a2e0 2024-08-15T17:59:05.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd37fe700 1 -- 172.21.15.38:0/2920394880 >> v2:172.21.15.38:3300/0 conn(0x7f0dd40b3b30 msgr2=0x7f0dd40b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:05.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd37fe700 1 --2- 172.21.15.38:0/2920394880 >> v2:172.21.15.38:3300/0 conn(0x7f0dd40b3b30 0x7f0dd40b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd37fe700 1 -- 172.21.15.38:0/2920394880 >> v1:172.21.15.38:6789/0 conn(0x7f0dd415a2e0 legacy=0x7f0dd41574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd27fc700 1 -- 172.21.15.38:0/2920394880 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2558255325 0 0) 0x7f0dc0003570 con 0x7f0dd415a2e0 2024-08-15T17:59:05.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd37fe700 1 -- 172.21.15.38:0/2920394880 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd415b2f0 con 0x7f0dd40aac50 2024-08-15T17:59:05.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd3fff700 1 --2- 172.21.15.38:0/2920394880 >> v2:172.21.15.38:3300/0 conn(0x7f0dd40b3b30 0x7f0dd40b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:05.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd37fe700 1 --2- 172.21.15.38:0/2920394880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd40aac50 0x7f0dd415aa30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0dd409e010 tx=0x7f0dcc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5559593ae47e5a33 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:05.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd27fc700 1 -- 172.21.15.38:0/2920394880 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc00e070 con 0x7f0dd40aac50 2024-08-15T17:59:05.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd27fc700 1 -- 172.21.15.38:0/2920394880 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc00b7a0 con 0x7f0dd40aac50 2024-08-15T17:59:05.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd27fc700 1 -- 172.21.15.38:0/2920394880 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc001470 con 0x7f0dd40aac50 2024-08-15T17:59:05.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/2920394880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd40aac50 msgr2=0x7f0dd415aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd9ce2700 1 --2- 172.21.15.38:0/2920394880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd40aac50 0x7f0dd415aa30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0dd409e010 tx=0x7f0dcc009780 comp rx=0 tx=0).stop 2024-08-15T17:59:05.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/2920394880 shutdown_connections 2024-08-15T17:59:05.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd9ce2700 1 --2- 172.21.15.38:0/2920394880 >> v2:172.21.15.38:3300/0 conn(0x7f0dd40b3b30 0x7f0dd40b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd9ce2700 1 --2- 172.21.15.38:0/2920394880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd40aac50 0x7f0dd415aa30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/2920394880 >> 172.21.15.38:0/2920394880 conn(0x7f0dd40994d0 msgr2=0x7f0dd40675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:05.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/2920394880 shutdown_connections 2024-08-15T17:59:05.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.745+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/2920394880 wait complete. 2024-08-15T17:59:05.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd9ce2700 1 Processor -- start 2024-08-15T17:59:05.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd9ce2700 1 -- start start 2024-08-15T17:59:05.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd9ce2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd40aac50 0x7f0dd4151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd9ce2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dd40b3b30 0x7f0dd40ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd9ce2700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0dd415a2e0 0x7f0dd40b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd9ce2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40ae9a0 con 0x7f0dd40b3b30 2024-08-15T17:59:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd9ce2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40add90 con 0x7f0dd40aac50 2024-08-15T17:59:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd9ce2700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40ae070 con 0x7f0dd415a2e0 2024-08-15T17:59:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dd40b3b30 0x7f0dd40ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dd40b3b30 0x7f0dd40ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38924/0 (socket says 172.21.15.38:38924) 2024-08-15T17:59:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd2ffd700 1 -- 172.21.15.38:0/4139203553 learned_addr learned my addr 172.21.15.38:0/4139203553 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd3fff700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0dd415a2e0 0x7f0dd40b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd37fe700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd40aac50 0x7f0dd4151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd2ffd700 1 -- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0dd415a2e0 msgr2=0x7f0dd40b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd2ffd700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0dd415a2e0 0x7f0dd40b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd2ffd700 1 -- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd40aac50 msgr2=0x7f0dd4151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd2ffd700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd40aac50 0x7f0dd4151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd2ffd700 1 -- 172.21.15.38:0/4139203553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc4009420 con 0x7f0dd40b3b30 2024-08-15T17:59:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd2ffd700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dd40b3b30 0x7f0dd40ad2b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00cb90 tx=0x7f0dbc001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dcb7fe700 1 -- 172.21.15.38:0/4139203553 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc001d40 con 0x7f0dd40b3b30 2024-08-15T17:59:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dcb7fe700 1 -- 172.21.15.38:0/4139203553 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc007360 con 0x7f0dd40b3b30 2024-08-15T17:59:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/4139203553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc009a20 con 0x7f0dd40b3b30 2024-08-15T17:59:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dcb7fe700 1 -- 172.21.15.38:0/4139203553 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc004830 con 0x7f0dd40b3b30 2024-08-15T17:59:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.749+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/4139203553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd40ae630 con 0x7f0dd40b3b30 2024-08-15T17:59:05.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.753+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/4139203553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd40aeb10 con 0x7f0dd40b3b30 2024-08-15T17:59:05.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.753+0000 7f0dcb7fe700 1 -- 172.21.15.38:0/4139203553 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0dbc004ba0 con 0x7f0dd40b3b30 2024-08-15T17:59:05.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.753+0000 7f0dcb7fe700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0db4060890 0x7f0db4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:05.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.753+0000 7f0dcb7fe700 1 -- 172.21.15.38:0/4139203553 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0dbc07c3b0 con 0x7f0dd40b3b30 2024-08-15T17:59:05.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.753+0000 7f0dd37fe700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0db4060890 0x7f0db4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:05.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.753+0000 7f0dd37fe700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0db4060890 0x7f0db4062d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0097c0 tx=0x7f0dcc018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:05.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.757+0000 7f0dcb7fe700 1 -- 172.21.15.38:0/4139203553 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0dbc04c880 con 0x7f0dd40b3b30 2024-08-15T17:59:05.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:05.981+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/4139203553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7f0dd40645c0 con 0x7f0dd40b3b30 2024-08-15T17:59:06.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.461+0000 7f0dcb7fe700 1 -- 172.21.15.38:0/4139203553 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0dbc0180d0 con 0x7f0dd40b3b30 2024-08-15T17:59:06.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0db4060890 msgr2=0x7f0db4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:06.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0db4060890 0x7f0db4062d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0097c0 tx=0x7f0dcc018000 comp rx=0 tx=0).stop 2024-08-15T17:59:06.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/4139203553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dd40b3b30 msgr2=0x7f0dd40ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:06.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dd40b3b30 0x7f0dd40ad2b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00cb90 tx=0x7f0dbc001ab0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/4139203553 shutdown_connections 2024-08-15T17:59:06.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f0db4060890 0x7f0db4062d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0dd415a2e0 0x7f0dd40b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dd40b3b30 0x7f0dd40ad2b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 --2- 172.21.15.38:0/4139203553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd40aac50 0x7f0dd4151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/4139203553 >> 172.21.15.38:0/4139203553 conn(0x7f0dd40994d0 msgr2=0x7f0dd40aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:06.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/4139203553 shutdown_connections 2024-08-15T17:59:06.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.465+0000 7f0dd9ce2700 1 -- 172.21.15.38:0/4139203553 wait complete. 2024-08-15T17:59:06.484 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-08-15T17:59:06.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.601+0000 7f3412e23700 1 Processor -- start 2024-08-15T17:59:06.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.601+0000 7f3412e23700 1 -- start start 2024-08-15T17:59:06.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f3412e23700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c156c00 0x7f340c157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:06.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f3412e23700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c0cadd0 con 0x7f340c0688d0 2024-08-15T17:59:06.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f3412e23700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c0cb0b0 con 0x7f340c068ca0 2024-08-15T17:59:06.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f3412e23700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0cb390 con 0x7f340c156c00 2024-08-15T17:59:06.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f3410bbf700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f340c0688d0 0x7f340c0cbe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35276/0 (socket says 172.21.15.38:35276) 2024-08-15T17:59:06.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f3410bbf700 1 -- 172.21.15.38:0/616641379 learned_addr learned my addr 172.21.15.38:0/616641379 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:06.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f34113c0700 1 --2- 172.21.15.38:0/616641379 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c156c00 0x7f340c157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:06.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f340b7fe700 1 -- 172.21.15.38:0/616641379 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153362603 0 0) 0x7f340c0cadd0 con 0x7f340c0688d0 2024-08-15T17:59:06.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f340b7fe700 1 -- 172.21.15.38:0/616641379 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33f4003570 con 0x7f340c0688d0 2024-08-15T17:59:06.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f340b7fe700 1 -- 172.21.15.38:0/616641379 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841866397 0 0) 0x7f340c0cb0b0 con 0x7f340c068ca0 2024-08-15T17:59:06.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f340b7fe700 1 -- 172.21.15.38:0/616641379 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f340c0cadd0 con 0x7f340c068ca0 2024-08-15T17:59:06.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f34113c0700 1 -- 172.21.15.38:0/616641379 >> v1:172.21.15.104:6789/0 conn(0x7f340c068ca0 legacy=0x7f340c0ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:06.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f34113c0700 1 -- 172.21.15.38:0/616641379 >> v1:172.21.15.38:6789/0 conn(0x7f340c0688d0 legacy=0x7f340c0cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:06.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f34113c0700 1 -- 172.21.15.38:0/616641379 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0cb670 con 0x7f340c156c00 2024-08-15T17:59:06.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f34113c0700 1 --2- 172.21.15.38:0/616641379 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c156c00 0x7f340c157000 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3404009e90 tx=0x7f3404009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13087166d278d1f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:06.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f340b7fe700 1 -- 172.21.15.38:0/616641379 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400e070 con 0x7f340c156c00 2024-08-15T17:59:06.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f340b7fe700 1 -- 172.21.15.38:0/616641379 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340400bb80 con 0x7f340c156c00 2024-08-15T17:59:06.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.605+0000 7f340b7fe700 1 -- 172.21.15.38:0/616641379 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404012430 con 0x7f340c156c00 2024-08-15T17:59:06.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.609+0000 7f3412e23700 1 -- 172.21.15.38:0/616641379 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c156c00 msgr2=0x7f340c157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:06.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.609+0000 7f3412e23700 1 --2- 172.21.15.38:0/616641379 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c156c00 0x7f340c157000 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3404009e90 tx=0x7f3404009780 comp rx=0 tx=0).stop 2024-08-15T17:59:06.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.609+0000 7f3412e23700 1 -- 172.21.15.38:0/616641379 shutdown_connections 2024-08-15T17:59:06.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.609+0000 7f3412e23700 1 --2- 172.21.15.38:0/616641379 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c156c00 0x7f340c157000 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.609+0000 7f3412e23700 1 -- 172.21.15.38:0/616641379 >> 172.21.15.38:0/616641379 conn(0x7f340c06e310 msgr2=0x7f340c06b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:06.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.609+0000 7f3412e23700 1 -- 172.21.15.38:0/616641379 shutdown_connections 2024-08-15T17:59:06.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.609+0000 7f3412e23700 1 -- 172.21.15.38:0/616641379 wait complete. 2024-08-15T17:59:06.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.609+0000 7f3412e23700 1 Processor -- start 2024-08-15T17:59:06.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.609+0000 7f3412e23700 1 -- start start 2024-08-15T17:59:06.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3412e23700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f340c0688d0 0x7f340c159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:06.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3412e23700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c068ca0 0x7f340c0acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:06.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3412e23700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340c156c00 0x7f340c14d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:06.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3412e23700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0ad130 con 0x7f340c0688d0 2024-08-15T17:59:06.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3412e23700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0ad400 con 0x7f340c156c00 2024-08-15T17:59:06.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3412e23700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0abc00 con 0x7f340c068ca0 2024-08-15T17:59:06.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3410bbf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f340c0688d0 0x7f340c159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:06.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3410bbf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f340c0688d0 0x7f340c159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35202/0 (socket says 172.21.15.38:35202) 2024-08-15T17:59:06.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3410bbf700 1 -- 172.21.15.38:0/744906145 learned_addr learned my addr 172.21.15.38:0/744906145 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:06.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f340bfff700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c068ca0 0x7f340c0acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:06.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f34113c0700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340c156c00 0x7f340c14d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:06.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f340bfff700 1 -- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340c156c00 msgr2=0x7f340c14d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:06.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f340bfff700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340c156c00 0x7f340c14d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f340bfff700 1 -- 172.21.15.38:0/744906145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f340c0688d0 msgr2=0x7f340c159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:06.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f340bfff700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f340c0688d0 0x7f340c159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f340bfff700 1 -- 172.21.15.38:0/744906145 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3404009a20 con 0x7f340c068ca0 2024-08-15T17:59:06.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3410bbf700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f340c0688d0 0x7f340c159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:06.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f340bfff700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c068ca0 0x7f340c0acbf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f33fc000c00 tx=0x7f33fc007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:06.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3408ff9700 1 -- 172.21.15.38:0/744906145 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc011590 con 0x7f340c068ca0 2024-08-15T17:59:06.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3408ff9700 1 -- 172.21.15.38:0/744906145 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc0116f0 con 0x7f340c068ca0 2024-08-15T17:59:06.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3412e23700 1 -- 172.21.15.38:0/744906145 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c0abee0 con 0x7f340c068ca0 2024-08-15T17:59:06.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3408ff9700 1 -- 172.21.15.38:0/744906145 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc011ab0 con 0x7f340c068ca0 2024-08-15T17:59:06.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.613+0000 7f3412e23700 1 -- 172.21.15.38:0/744906145 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340c0ad6e0 con 0x7f340c068ca0 2024-08-15T17:59:06.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.617+0000 7f3412e23700 1 -- 172.21.15.38:0/744906145 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340c0ac3e0 con 0x7f340c068ca0 2024-08-15T17:59:06.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.617+0000 7f3408ff9700 1 -- 172.21.15.38:0/744906145 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f33fc01c460 con 0x7f340c068ca0 2024-08-15T17:59:06.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.617+0000 7f3408ff9700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f33f8060860 0x7f33f8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:06.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.617+0000 7f3408ff9700 1 -- 172.21.15.38:0/744906145 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f33fc07c7a0 con 0x7f340c068ca0 2024-08-15T17:59:06.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.617+0000 7f3410bbf700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f33f8060860 0x7f33f8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:06.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.617+0000 7f3410bbf700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f33f8060860 0x7f33f8062d10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f340c164fc0 tx=0x7f3400009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:06.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.629+0000 7f3408ff9700 1 -- 172.21.15.38:0/744906145 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f33fc04cb50 con 0x7f340c068ca0 2024-08-15T17:59:06.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.797+0000 7f3412e23700 1 -- 172.21.15.38:0/744906145 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f340c0645c0 con 0x7f340c068ca0 2024-08-15T17:59:06.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.805+0000 7f3408ff9700 1 -- 172.21.15.38:0/744906145 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f33fc050170 con 0x7f340c068ca0 2024-08-15T17:59:06.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 -- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f33f8060860 msgr2=0x7f33f8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:06.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f33f8060860 0x7f33f8062d10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f340c164fc0 tx=0x7f3400009040 comp rx=0 tx=0).stop 2024-08-15T17:59:06.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 -- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c068ca0 msgr2=0x7f340c0acbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:06.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c068ca0 0x7f340c0acbf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f33fc000c00 tx=0x7f33fc007d30 comp rx=0 tx=0).stop 2024-08-15T17:59:06.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 -- 172.21.15.38:0/744906145 shutdown_connections 2024-08-15T17:59:06.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f33f8060860 0x7f33f8062d10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340c156c00 0x7f340c14d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f340c068ca0 0x7f340c0acbf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 --2- 172.21.15.38:0/744906145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f340c0688d0 0x7f340c159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:06.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 -- 172.21.15.38:0/744906145 >> 172.21.15.38:0/744906145 conn(0x7f340c06e310 msgr2=0x7f340c0b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:06.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 -- 172.21.15.38:0/744906145 shutdown_connections 2024-08-15T17:59:06.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:06.809+0000 7f3412e23700 1 -- 172.21.15.38:0/744906145 wait complete. 2024-08-15T17:59:06.825 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-08-15T17:59:06.825 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T17:59:06.839 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T17:59:06.859 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-08-15T17:59:06.859 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-08-15T17:59:06.859 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:59:06.859 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-08-15T17:59:06.895 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:59:06.895 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T17:59:06.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:06.895 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-15T17:59:06.951 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2024-08-15T17:59:06.951 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:59:06.952 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-08-15T17:59:06.981 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:59:06.981 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T17:59:06.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:06.981 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-08-15T17:59:07.041 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-08-15T17:59:07.041 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:59:07.041 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-08-15T17:59:07.059 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:59:07.059 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T17:59:07.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:07.060 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-08-15T17:59:07.066 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2024-08-15T17:59:07.066 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:59:07.066 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-08-15T17:59:07.082 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:59:07.083 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T17:59:07.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:07.083 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-08-15T17:59:07.132 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-08-15T17:59:07.133 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:59:07.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.245+0000 7f94409bb700 1 Processor -- start 2024-08-15T17:59:07.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.245+0000 7f94409bb700 1 -- start start 2024-08-15T17:59:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f943c15aa30 0x7f943c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c0b3b30 0x7f943c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c15a820 con 0x7f943c0aac50 2024-08-15T17:59:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943c15bc90 con 0x7f943c15aa30 2024-08-15T17:59:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0b4490 con 0x7f943c0b3b30 2024-08-15T17:59:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943a59c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f943c0aac50 0x7f943c1574a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35286/0 (socket says 172.21.15.38:35286) 2024-08-15T17:59:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943a59c700 1 -- 172.21.15.38:0/703390686 learned_addr learned my addr 172.21.15.38:0/703390686 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943ad9d700 1 --2- 172.21.15.38:0/703390686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c0b3b30 0x7f943c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f9439d9b700 1 --2- 172.21.15.38:0/703390686 >> v2:172.21.15.104:3300/0 conn(0x7f943c15aa30 0x7f943c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943959a700 1 -- 172.21.15.38:0/703390686 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302630849 0 0) 0x7f9430001290 con 0x7f943c0aac50 2024-08-15T17:59:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943959a700 1 -- 172.21.15.38:0/703390686 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9424003570 con 0x7f943c0aac50 2024-08-15T17:59:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943ad9d700 1 -- 172.21.15.38:0/703390686 >> v2:172.21.15.104:3300/0 conn(0x7f943c15aa30 msgr2=0x7f943c15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943ad9d700 1 --2- 172.21.15.38:0/703390686 >> v2:172.21.15.104:3300/0 conn(0x7f943c15aa30 0x7f943c15a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943ad9d700 1 -- 172.21.15.38:0/703390686 >> v1:172.21.15.38:6789/0 conn(0x7f943c0aac50 legacy=0x7f943c1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943ad9d700 1 -- 172.21.15.38:0/703390686 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c15b010 con 0x7f943c0b3b30 2024-08-15T17:59:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f9439d9b700 1 --2- 172.21.15.38:0/703390686 >> v2:172.21.15.104:3300/0 conn(0x7f943c15aa30 0x7f943c15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943ad9d700 1 --2- 172.21.15.38:0/703390686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c0b3b30 0x7f943c0b3f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9434009e60 tx=0x7f9434009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=669cb6ac5ac88337 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:07.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943959a700 1 -- 172.21.15.38:0/703390686 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400e070 con 0x7f943c0b3b30 2024-08-15T17:59:07.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943959a700 1 -- 172.21.15.38:0/703390686 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943400bab0 con 0x7f943c0b3b30 2024-08-15T17:59:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f943959a700 1 -- 172.21.15.38:0/703390686 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400be70 con 0x7f943c0b3b30 2024-08-15T17:59:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 -- 172.21.15.38:0/703390686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c0b3b30 msgr2=0x7f943c0b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 --2- 172.21.15.38:0/703390686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c0b3b30 0x7f943c0b3f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9434009e60 tx=0x7f9434009780 comp rx=0 tx=0).stop 2024-08-15T17:59:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 -- 172.21.15.38:0/703390686 shutdown_connections 2024-08-15T17:59:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 --2- 172.21.15.38:0/703390686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c0b3b30 0x7f943c0b3f50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 --2- 172.21.15.38:0/703390686 >> v2:172.21.15.104:3300/0 conn(0x7f943c15aa30 0x7f943c15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 -- 172.21.15.38:0/703390686 >> 172.21.15.38:0/703390686 conn(0x7f943c0994d0 msgr2=0x7f943c0675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.249+0000 7f94409bb700 1 -- 172.21.15.38:0/703390686 shutdown_connections 2024-08-15T17:59:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f94409bb700 1 -- 172.21.15.38:0/703390686 wait complete. 2024-08-15T17:59:07.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f94409bb700 1 Processor -- start 2024-08-15T17:59:07.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f94409bb700 1 -- start start 2024-08-15T17:59:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f94409bb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f943c0aac50 0x7f943c168770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f94409bb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943c0b3b30 0x7f943c09d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f94409bb700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c15aa30 0x7f943c099e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f94409bb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0af360 con 0x7f943c0b3b30 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f94409bb700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0af640 con 0x7f943c0aac50 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f94409bb700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c09c980 con 0x7f943c15aa30 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f9439d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943c0b3b30 0x7f943c09d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f9439d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943c0b3b30 0x7f943c09d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35208/0 (socket says 172.21.15.38:35208) 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f9439d9b700 1 -- 172.21.15.38:0/3017358739 learned_addr learned my addr 172.21.15.38:0/3017358739 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f943a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f943c0aac50 0x7f943c168770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.253+0000 7f943ad9d700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c15aa30 0x7f943c099e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f9439d9b700 1 -- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c15aa30 msgr2=0x7f943c099e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f9439d9b700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c15aa30 0x7f943c099e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f9439d9b700 1 -- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f943c0aac50 msgr2=0x7f943c168770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f9439d9b700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f943c0aac50 0x7f943c168770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f9439d9b700 1 -- 172.21.15.38:0/3017358739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c009420 con 0x7f943c0b3b30 2024-08-15T17:59:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f943ad9d700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c15aa30 0x7f943c099e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f9439d9b700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943c0b3b30 0x7f943c09d460 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f942c00ec80 tx=0x7f942c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f942a7fc700 1 -- 172.21.15.38:0/3017358739 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c00c850 con 0x7f943c0b3b30 2024-08-15T17:59:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f942a7fc700 1 -- 172.21.15.38:0/3017358739 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c00c9b0 con 0x7f943c0b3b30 2024-08-15T17:59:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f94409bb700 1 -- 172.21.15.38:0/3017358739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9434009a20 con 0x7f943c0b3b30 2024-08-15T17:59:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f942a7fc700 1 -- 172.21.15.38:0/3017358739 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c00cd30 con 0x7f943c0b3b30 2024-08-15T17:59:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f94409bb700 1 -- 172.21.15.38:0/3017358739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c09cf40 con 0x7f943c0b3b30 2024-08-15T17:59:07.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f942a7fc700 1 -- 172.21.15.38:0/3017358739 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f942c013680 con 0x7f943c0b3b30 2024-08-15T17:59:07.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f94409bb700 1 -- 172.21.15.38:0/3017358739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c0645c0 con 0x7f943c0b3b30 2024-08-15T17:59:07.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f942a7fc700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f941c060860 0x7f941c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f942a7fc700 1 -- 172.21.15.38:0/3017358739 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f942c07d1e0 con 0x7f943c0b3b30 2024-08-15T17:59:07.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.257+0000 7f943a59c700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f941c060860 0x7f941c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.261+0000 7f943a59c700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f941c060860 0x7f941c062d10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f94300013f0 tx=0x7f9430009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:07.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.265+0000 7f942a7fc700 1 -- 172.21.15.38:0/3017358739 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f942c0510c0 con 0x7f943c0b3b30 2024-08-15T17:59:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.465+0000 7f94409bb700 1 -- 172.21.15.38:0/3017358739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f943c0b1fd0 con 0x7f943c0b3b30 2024-08-15T17:59:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.465+0000 7f942a7fc700 1 -- 172.21.15.38:0/3017358739 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f942c050c50 con 0x7f943c0b3b30 2024-08-15T17:59:07.469 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:07.470 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2444719791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2444719791},{"type":"v1","addr":"172.21.15.38:6833","nonce":2444719791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2667173752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2667173752},{"type":"v1","addr":"172.21.15.38:6835","nonce":2667173752}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1185013072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1185013072},{"type":"v1","addr":"172.21.15.104:6835","nonce":1185013072}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:58:35.622492+0000","modified":"2024-08-15T17:58:38.885045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/3628618356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3628618356},{"type":"v1","addr":"172.21.15.104:6837","nonce":3628618356}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:59:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 -- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f941c060860 msgr2=0x7f941c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f941c060860 0x7f941c062d10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f94300013f0 tx=0x7f9430009040 comp rx=0 tx=0).stop 2024-08-15T17:59:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 -- 172.21.15.38:0/3017358739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943c0b3b30 msgr2=0x7f943c09d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943c0b3b30 0x7f943c09d460 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f942c00ec80 tx=0x7f942c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 -- 172.21.15.38:0/3017358739 shutdown_connections 2024-08-15T17:59:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f941c060860 0x7f941c062d10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f943c15aa30 0x7f943c099e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f943c0b3b30 0x7f943c09d460 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 --2- 172.21.15.38:0/3017358739 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f943c0aac50 0x7f943c168770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 -- 172.21.15.38:0/3017358739 >> 172.21.15.38:0/3017358739 conn(0x7f943c0994d0 msgr2=0x7f943c0b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 -- 172.21.15.38:0/3017358739 shutdown_connections 2024-08-15T17:59:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.469+0000 7f94409bb700 1 -- 172.21.15.38:0/3017358739 wait complete. 2024-08-15T17:59:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-08-15T17:59:07.483 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:59:07.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f14973ea700 1 Processor -- start 2024-08-15T17:59:07.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f14973ea700 1 -- start start 2024-08-15T17:59:07.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f14973ea700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f149015d540 0x7f14900af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f14973ea700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f14900a9430 0x7f14900b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f14973ea700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14900b0270 con 0x7f149006e830 2024-08-15T17:59:07.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f14973ea700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14900af660 con 0x7f149015d540 2024-08-15T17:59:07.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f14973ea700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14900af940 con 0x7f14900a9430 2024-08-15T17:59:07.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1494985700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f149006e830 0x7f14900a8d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35294/0 (socket says 172.21.15.38:35294) 2024-08-15T17:59:07.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1494985700 1 -- 172.21.15.38:0/3143687265 learned_addr learned my addr 172.21.15.38:0/3143687265 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:07.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1495186700 1 --2- 172.21.15.38:0/3143687265 >> v2:172.21.15.38:3300/0 conn(0x7f149015d540 0x7f14900af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1495987700 1 --2- 172.21.15.38:0/3143687265 >> v2:172.21.15.104:3300/0 conn(0x7f14900a9430 0x7f14900b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922426284 0 0) 0x7f14900b0270 con 0x7f149006e830 2024-08-15T17:59:07.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1478003570 con 0x7f149006e830 2024-08-15T17:59:07.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2653605835 0 0) 0x7f1478003570 con 0x7f149006e830 2024-08-15T17:59:07.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 >> v2:172.21.15.104:3300/0 conn(0x7f14900a9430 msgr2=0x7f14900b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:07.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 --2- 172.21.15.38:0/3143687265 >> v2:172.21.15.104:3300/0 conn(0x7f14900a9430 0x7f14900b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 >> v2:172.21.15.38:3300/0 conn(0x7f149015d540 msgr2=0x7f14900af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:07.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 --2- 172.21.15.38:0/3143687265 >> v2:172.21.15.38:3300/0 conn(0x7f149015d540 0x7f14900af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14900afc20 con 0x7f149006e830 2024-08-15T17:59:07.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f14880017f0 con 0x7f149006e830 2024-08-15T17:59:07.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 >> v1:172.21.15.38:6789/0 conn(0x7f149006e830 legacy=0x7f14900a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1478003a60 0x7f1478003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1478004810 0x7f1478004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14780051c0 0x7f147800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900af660 con 0x7f1478004810 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900af940 con 0x7f14780051c0 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147800bfd0 con 0x7f1478003a60 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1494985700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1478004810 0x7f1478004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1495987700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14780051c0 0x7f147800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1495186700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1478003a60 0x7f1478003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1494985700 1 -- 172.21.15.38:0/3143687265 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1478003a60 msgr2=0x7f1478003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1494985700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1478003a60 0x7f1478003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1494985700 1 -- 172.21.15.38:0/3143687265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14780051c0 msgr2=0x7f147800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1494985700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14780051c0 0x7f147800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1494985700 1 -- 172.21.15.38:0/3143687265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1478003570 con 0x7f1478004810 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1495987700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14780051c0 0x7f147800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:07.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1495987700 1 -- 172.21.15.38:0/3143687265 reap_dead start 2024-08-15T17:59:07.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1495186700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1478003a60 0x7f1478003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:07.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.585+0000 7f1494985700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1478004810 0x7f1478004c80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f148800a940 tx=0x7f1488002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82acf7d048faad2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:07.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488012440 con 0x7f1478004810 2024-08-15T17:59:07.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14880125c0 con 0x7f1478004810 2024-08-15T17:59:07.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f1487fff700 1 -- 172.21.15.38:0/3143687265 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488012980 con 0x7f1478004810 2024-08-15T17:59:07.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f14973ea700 1 -- 172.21.15.38:0/3143687265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1478004810 msgr2=0x7f1478004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f14973ea700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1478004810 0x7f1478004c80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f148800a940 tx=0x7f1488002210 comp rx=0 tx=0).stop 2024-08-15T17:59:07.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f14973ea700 1 -- 172.21.15.38:0/3143687265 shutdown_connections 2024-08-15T17:59:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f14973ea700 1 --2- 172.21.15.38:0/3143687265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1478004810 0x7f1478004c80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f14973ea700 1 -- 172.21.15.38:0/3143687265 >> 172.21.15.38:0/3143687265 conn(0x7f1490066e00 msgr2=0x7f1490067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f14973ea700 1 -- 172.21.15.38:0/3143687265 shutdown_connections 2024-08-15T17:59:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f14973ea700 1 -- 172.21.15.38:0/3143687265 wait complete. 2024-08-15T17:59:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f14973ea700 1 Processor -- start 2024-08-15T17:59:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.589+0000 7f14973ea700 1 -- start start 2024-08-15T17:59:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f14973ea700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1478004810 0x7f149014d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f14973ea700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14900aba80 0x7f14900ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f14973ea700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14900cae00 0x7f14900ac560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f14973ea700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149015d910 con 0x7f14900cae00 2024-08-15T17:59:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f14973ea700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900cb270 con 0x7f14900aba80 2024-08-15T17:59:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f14973ea700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14900ad220 con 0x7f1478004810 2024-08-15T17:59:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f1495987700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14900cae00 0x7f14900ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f1495987700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14900cae00 0x7f14900ac560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35246/0 (socket says 172.21.15.38:35246) 2024-08-15T17:59:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f1495987700 1 -- 172.21.15.38:0/3186719361 learned_addr learned my addr 172.21.15.38:0/3186719361 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f1495186700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1478004810 0x7f149014d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f1494985700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14900aba80 0x7f14900ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f1495987700 1 -- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1478004810 msgr2=0x7f149014d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f1495987700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1478004810 0x7f149014d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f1495987700 1 -- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14900aba80 msgr2=0x7f14900ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f1495987700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14900aba80 0x7f14900ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f1495987700 1 -- 172.21.15.38:0/3186719361 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14900ab5f0 con 0x7f14900cae00 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f1495987700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14900cae00 0x7f14900ac560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1480011590 tx=0x7f1480008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f14857fa700 1 -- 172.21.15.38:0/3186719361 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480015670 con 0x7f14900cae00 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f14857fa700 1 -- 172.21.15.38:0/3186719361 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14800157f0 con 0x7f14900cae00 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f14973ea700 1 -- 172.21.15.38:0/3186719361 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14900aacc0 con 0x7f14900cae00 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f14857fa700 1 -- 172.21.15.38:0/3186719361 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480015bb0 con 0x7f14900cae00 2024-08-15T17:59:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.593+0000 7f14973ea700 1 -- 172.21.15.38:0/3186719361 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14900ab280 con 0x7f14900cae00 2024-08-15T17:59:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.597+0000 7f14973ea700 1 -- 172.21.15.38:0/3186719361 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14900ab780 con 0x7f14900cae00 2024-08-15T17:59:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.597+0000 7f14857fa700 1 -- 172.21.15.38:0/3186719361 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1480020460 con 0x7f14900cae00 2024-08-15T17:59:07.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.597+0000 7f14857fa700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f147c060890 0x7f147c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.597+0000 7f14857fa700 1 -- 172.21.15.38:0/3186719361 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f148007cbb0 con 0x7f14900cae00 2024-08-15T17:59:07.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.597+0000 7f1495186700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f147c060890 0x7f147c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.597+0000 7f1495186700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f147c060890 0x7f147c062d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f14900ae540 tx=0x7f148c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:07.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.601+0000 7f14857fa700 1 -- 172.21.15.38:0/3186719361 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f148004d000 con 0x7f14900cae00 2024-08-15T17:59:07.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.769+0000 7f14973ea700 1 -- 172.21.15.38:0/3186719361 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14900645c0 con 0x7f14900cae00 2024-08-15T17:59:07.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.769+0000 7f14857fa700 1 -- 172.21.15.38:0/3186719361 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f1480050620 con 0x7f14900cae00 2024-08-15T17:59:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:07.774 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:59:07.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 -- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f147c060890 msgr2=0x7f147c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f147c060890 0x7f147c062d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f14900ae540 tx=0x7f148c008040 comp rx=0 tx=0).stop 2024-08-15T17:59:07.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 -- 172.21.15.38:0/3186719361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14900cae00 msgr2=0x7f14900ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14900cae00 0x7f14900ac560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1480011590 tx=0x7f1480008040 comp rx=0 tx=0).stop 2024-08-15T17:59:07.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 -- 172.21.15.38:0/3186719361 shutdown_connections 2024-08-15T17:59:07.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f147c060890 0x7f147c062d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14900cae00 0x7f14900ac560 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14900aba80 0x7f14900ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 --2- 172.21.15.38:0/3186719361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1478004810 0x7f149014d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 -- 172.21.15.38:0/3186719361 >> 172.21.15.38:0/3186719361 conn(0x7f1490066e00 msgr2=0x7f14900676e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:07.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 -- 172.21.15.38:0/3186719361 shutdown_connections 2024-08-15T17:59:07.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.773+0000 7f14973ea700 1 -- 172.21.15.38:0/3186719361 wait complete. 2024-08-15T17:59:07.789 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:58:34.079267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:58:34.621969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:59:07.790 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:59:07.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.901+0000 7f84271f7700 1 Processor -- start 2024-08-15T17:59:07.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.901+0000 7f84271f7700 1 -- start start 2024-08-15T17:59:07.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f84201554c0 0x7f84200699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8420069ee0 0x7f84200686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842006b000 con 0x7f8420155890 2024-08-15T17:59:07.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842006a3f0 con 0x7f8420069ee0 2024-08-15T17:59:07.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842006a6d0 con 0x7f84201554c0 2024-08-15T17:59:07.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f8424f93700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f84201554c0 0x7f84200699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f841ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8420069ee0 0x7f84200686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f8424f93700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f84201554c0 0x7f84200699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:59240/0 (socket says 172.21.15.38:59240) 2024-08-15T17:59:07.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f8424f93700 1 -- 172.21.15.38:0/2708405749 learned_addr learned my addr 172.21.15.38:0/2708405749 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:07.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f841ffff700 1 -- 172.21.15.38:0/2708405749 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f84201554c0 msgr2=0x7f84200699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:07.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f841ffff700 1 --2- 172.21.15.38:0/2708405749 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f84201554c0 0x7f84200699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f841ffff700 1 -- 172.21.15.38:0/2708405749 >> v1:172.21.15.104:6789/0 conn(0x7f8420155890 legacy=0x7f8420159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f841ffff700 1 -- 172.21.15.38:0/2708405749 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842006a9b0 con 0x7f8420069ee0 2024-08-15T17:59:07.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f841ffff700 1 --2- 172.21.15.38:0/2708405749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420069ee0 0x7f84200686a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f840c0098e0 tx=0x7f840c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb85b6f9814c5458 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:07.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f841f7fe700 1 -- 172.21.15.38:0/2708405749 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00b640 con 0x7f8420069ee0 2024-08-15T17:59:07.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f841f7fe700 1 -- 172.21.15.38:0/2708405749 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c00b7a0 con 0x7f8420069ee0 2024-08-15T17:59:07.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f841f7fe700 1 -- 172.21.15.38:0/2708405749 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00bb40 con 0x7f8420069ee0 2024-08-15T17:59:07.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 -- 172.21.15.38:0/2708405749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420069ee0 msgr2=0x7f84200686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 --2- 172.21.15.38:0/2708405749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420069ee0 0x7f84200686a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f840c0098e0 tx=0x7f840c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 -- 172.21.15.38:0/2708405749 shutdown_connections 2024-08-15T17:59:07.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 --2- 172.21.15.38:0/2708405749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420069ee0 0x7f84200686a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 --2- 172.21.15.38:0/2708405749 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f84201554c0 0x7f84200699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 -- 172.21.15.38:0/2708405749 >> 172.21.15.38:0/2708405749 conn(0x7f84200c9bf0 msgr2=0x7f842006d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:07.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 -- 172.21.15.38:0/2708405749 shutdown_connections 2024-08-15T17:59:07.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.905+0000 7f84271f7700 1 -- 172.21.15.38:0/2708405749 wait complete. 2024-08-15T17:59:07.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f84271f7700 1 Processor -- start 2024-08-15T17:59:07.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f84271f7700 1 -- start start 2024-08-15T17:59:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f84271f7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8420069ee0 0x7f8420156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f84271f7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84201554c0 0x7f84201410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f84271f7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420155890 0x7f84200b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f84271f7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8420157520 con 0x7f84201554c0 2024-08-15T17:59:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f84271f7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8420157910 con 0x7f8420155890 2024-08-15T17:59:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f84271f7700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8420157bf0 con 0x7f8420069ee0 2024-08-15T17:59:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f841ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84201554c0 0x7f84201410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f841ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84201554c0 0x7f84201410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35254/0 (socket says 172.21.15.38:35254) 2024-08-15T17:59:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f8424f93700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8420069ee0 0x7f8420156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f841ffff700 1 -- 172.21.15.38:0/3057649724 learned_addr learned my addr 172.21.15.38:0/3057649724 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.909+0000 7f8425794700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420155890 0x7f84200b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f841ffff700 1 -- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8420069ee0 msgr2=0x7f8420156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f841ffff700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8420069ee0 0x7f8420156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f841ffff700 1 -- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420155890 msgr2=0x7f84200b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:07.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f841ffff700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420155890 0x7f84200b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:07.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f841ffff700 1 -- 172.21.15.38:0/3057649724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c009420 con 0x7f84201554c0 2024-08-15T17:59:07.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f841ffff700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84201554c0 0x7f84201410e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f840c00a010 tx=0x7f840c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:07.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f841cff9700 1 -- 172.21.15.38:0/3057649724 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c01bc80 con 0x7f84201554c0 2024-08-15T17:59:07.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f841cff9700 1 -- 172.21.15.38:0/3057649724 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c01be20 con 0x7f84201554c0 2024-08-15T17:59:07.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f84271f7700 1 -- 172.21.15.38:0/3057649724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8420157ed0 con 0x7f84201554c0 2024-08-15T17:59:07.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f841cff9700 1 -- 172.21.15.38:0/3057649724 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c011670 con 0x7f84201554c0 2024-08-15T17:59:07.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f84271f7700 1 -- 172.21.15.38:0/3057649724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8420156cc0 con 0x7f84201554c0 2024-08-15T17:59:07.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f84271f7700 1 -- 172.21.15.38:0/3057649724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84200645c0 con 0x7f84201554c0 2024-08-15T17:59:07.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.913+0000 7f841cff9700 1 -- 172.21.15.38:0/3057649724 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f840c011aa0 con 0x7f84201554c0 2024-08-15T17:59:07.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.917+0000 7f841cff9700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8404060860 0x7f8404062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:07.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.917+0000 7f841cff9700 1 -- 172.21.15.38:0/3057649724 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f840c07e270 con 0x7f84201554c0 2024-08-15T17:59:07.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.917+0000 7f8424f93700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8404060860 0x7f8404062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:07.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.917+0000 7f8424f93700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8404060860 0x7f8404062d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8414006fd0 tx=0x7f8414009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:07.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:07.921+0000 7f841cff9700 1 -- 172.21.15.38:0/3057649724 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f840c04e6c0 con 0x7f84201554c0 2024-08-15T17:59:08.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.085+0000 7f84271f7700 1 -- 172.21.15.38:0/3057649724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f842006e870 con 0x7f84201554c0 2024-08-15T17:59:08.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.085+0000 7f841cff9700 1 -- 172.21.15.38:0/3057649724 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f840c015020 con 0x7f84201554c0 2024-08-15T17:59:08.089 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T17:59:08.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 -- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8404060860 msgr2=0x7f8404062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8404060860 0x7f8404062d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8414006fd0 tx=0x7f8414009250 comp rx=0 tx=0).stop 2024-08-15T17:59:08.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 -- 172.21.15.38:0/3057649724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84201554c0 msgr2=0x7f84201410e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84201554c0 0x7f84201410e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f840c00a010 tx=0x7f840c017000 comp rx=0 tx=0).stop 2024-08-15T17:59:08.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 -- 172.21.15.38:0/3057649724 shutdown_connections 2024-08-15T17:59:08.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8404060860 0x7f8404062d10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420155890 0x7f84200b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84201554c0 0x7f84201410e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 --2- 172.21.15.38:0/3057649724 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8420069ee0 0x7f8420156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 -- 172.21.15.38:0/3057649724 >> 172.21.15.38:0/3057649724 conn(0x7f84200c9bf0 msgr2=0x7f84200ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:08.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 -- 172.21.15.38:0/3057649724 shutdown_connections 2024-08-15T17:59:08.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.089+0000 7f84271f7700 1 -- 172.21.15.38:0/3057649724 wait complete. 2024-08-15T17:59:08.103 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:59:08.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.221+0000 7f89f685d700 1 Processor -- start 2024-08-15T17:59:08.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.221+0000 7f89f685d700 1 -- start start 2024-08-15T17:59:08.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.221+0000 7f89f685d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f89f00c9260 0x7f89f00ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.221+0000 7f89f685d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f0159b90 0x7f89f0069e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.221+0000 7f89f685d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f0157580 con 0x7f89f0153ba0 2024-08-15T17:59:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.221+0000 7f89f685d700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f0153f70 con 0x7f89f00c9260 2024-08-15T17:59:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.221+0000 7f89f685d700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0156970 con 0x7f89f0159b90 2024-08-15T17:59:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.221+0000 7f89effff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f89f0153ba0 0x7f89f00c8b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35310/0 (socket says 172.21.15.38:35310) 2024-08-15T17:59:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.221+0000 7f89effff700 1 -- 172.21.15.38:0/1982192048 learned_addr learned my addr 172.21.15.38:0/1982192048 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89ef7fe700 1 --2- 172.21.15.38:0/1982192048 >> v2:172.21.15.104:3300/0 conn(0x7f89f00c9260 0x7f89f00ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89f4dfa700 1 --2- 172.21.15.38:0/1982192048 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f0159b90 0x7f89f0069e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89eeffd700 1 -- 172.21.15.38:0/1982192048 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109574462 0 0) 0x7f89f0157580 con 0x7f89f0153ba0 2024-08-15T17:59:08.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89eeffd700 1 -- 172.21.15.38:0/1982192048 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89dc003570 con 0x7f89f0153ba0 2024-08-15T17:59:08.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89ef7fe700 1 -- 172.21.15.38:0/1982192048 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f0159b90 msgr2=0x7f89f0069e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:08.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89ef7fe700 1 --2- 172.21.15.38:0/1982192048 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f0159b90 0x7f89f0069e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89ef7fe700 1 -- 172.21.15.38:0/1982192048 >> v1:172.21.15.38:6789/0 conn(0x7f89f0153ba0 legacy=0x7f89f00c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89ef7fe700 1 -- 172.21.15.38:0/1982192048 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0156c50 con 0x7f89f00c9260 2024-08-15T17:59:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89ef7fe700 1 --2- 172.21.15.38:0/1982192048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f00c9260 0x7f89f00ca480 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f89d80098e0 tx=0x7f89d80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc8bb04ffbc6c41 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89eeffd700 1 -- 172.21.15.38:0/1982192048 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8011630 con 0x7f89f00c9260 2024-08-15T17:59:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89eeffd700 1 -- 172.21.15.38:0/1982192048 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d8011790 con 0x7f89f00c9260 2024-08-15T17:59:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89eeffd700 1 -- 172.21.15.38:0/1982192048 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8011b10 con 0x7f89f00c9260 2024-08-15T17:59:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89f685d700 1 -- 172.21.15.38:0/1982192048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f00c9260 msgr2=0x7f89f00ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89f685d700 1 --2- 172.21.15.38:0/1982192048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f00c9260 0x7f89f00ca480 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f89d80098e0 tx=0x7f89d80091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89f685d700 1 -- 172.21.15.38:0/1982192048 shutdown_connections 2024-08-15T17:59:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89f685d700 1 --2- 172.21.15.38:0/1982192048 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f0159b90 0x7f89f0069e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89f685d700 1 --2- 172.21.15.38:0/1982192048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f00c9260 0x7f89f00ca480 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89f685d700 1 -- 172.21.15.38:0/1982192048 >> 172.21.15.38:0/1982192048 conn(0x7f89f00cd9c0 msgr2=0x7f89f00c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89f685d700 1 -- 172.21.15.38:0/1982192048 shutdown_connections 2024-08-15T17:59:08.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.225+0000 7f89f685d700 1 -- 172.21.15.38:0/1982192048 wait complete. 2024-08-15T17:59:08.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89f685d700 1 Processor -- start 2024-08-15T17:59:08.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89f685d700 1 -- start start 2024-08-15T17:59:08.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89f685d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f00c9260 0x7f89f014f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89f685d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f0153ba0 0x7f89f0169650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89f685d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f0159b90 0x7f89f00c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89f685d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0154500 con 0x7f89f00c9260 2024-08-15T17:59:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89f685d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0157ef0 con 0x7f89f0153ba0 2024-08-15T17:59:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89f685d700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f01541d0 con 0x7f89f0159b90 2024-08-15T17:59:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89effff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f00c9260 0x7f89f014f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89effff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f00c9260 0x7f89f014f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35256/0 (socket says 172.21.15.38:35256) 2024-08-15T17:59:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89effff700 1 -- 172.21.15.38:0/3323962888 learned_addr learned my addr 172.21.15.38:0/3323962888 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89ef7fe700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f0153ba0 0x7f89f0169650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89f4dfa700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f0159b90 0x7f89f00c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89effff700 1 -- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f0159b90 msgr2=0x7f89f00c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89effff700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f0159b90 0x7f89f00c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89effff700 1 -- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f0153ba0 msgr2=0x7f89f0169650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89effff700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f0153ba0 0x7f89f0169650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89effff700 1 -- 172.21.15.38:0/3323962888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8009420 con 0x7f89f00c9260 2024-08-15T17:59:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89ef7fe700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f0153ba0 0x7f89f0169650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89effff700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f00c9260 0x7f89f014f7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f89e800cd30 tx=0x7f89e8001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89e7fff700 1 -- 172.21.15.38:0/3323962888 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e800ce90 con 0x7f89f00c9260 2024-08-15T17:59:08.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89e7fff700 1 -- 172.21.15.38:0/3323962888 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e800aa20 con 0x7f89f00c9260 2024-08-15T17:59:08.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89f685d700 1 -- 172.21.15.38:0/3323962888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f016a000 con 0x7f89f00c9260 2024-08-15T17:59:08.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89e7fff700 1 -- 172.21.15.38:0/3323962888 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e800adc0 con 0x7f89f00c9260 2024-08-15T17:59:08.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.229+0000 7f89f685d700 1 -- 172.21.15.38:0/3323962888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f016a5c0 con 0x7f89f00c9260 2024-08-15T17:59:08.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.233+0000 7f89f685d700 1 -- 172.21.15.38:0/3323962888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f00645c0 con 0x7f89f00c9260 2024-08-15T17:59:08.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.233+0000 7f89e7fff700 1 -- 172.21.15.38:0/3323962888 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f89e8007300 con 0x7f89f00c9260 2024-08-15T17:59:08.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.233+0000 7f89e7fff700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89d0060860 0x7f89d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.233+0000 7f89e7fff700 1 -- 172.21.15.38:0/3323962888 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f89e807cf90 con 0x7f89f00c9260 2024-08-15T17:59:08.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.233+0000 7f89ef7fe700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89d0060860 0x7f89d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.237+0000 7f89ef7fe700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89d0060860 0x7f89d0062d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f89d80098e0 tx=0x7f89d8017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:08.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.241+0000 7f89e7fff700 1 -- 172.21.15.38:0/3323962888 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f89e804d3c0 con 0x7f89f00c9260 2024-08-15T17:59:08.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.413+0000 7f89f685d700 1 -- 172.21.15.38:0/3323962888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89f00cc790 con 0x7f89f00c9260 2024-08-15T17:59:08.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.413+0000 7f89e7fff700 1 -- 172.21.15.38:0/3323962888 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f89e80509e0 con 0x7f89f00c9260 2024-08-15T17:59:08.417 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T17:59:08.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 -- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89d0060860 msgr2=0x7f89d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89d0060860 0x7f89d0062d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f89d80098e0 tx=0x7f89d8017000 comp rx=0 tx=0).stop 2024-08-15T17:59:08.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 -- 172.21.15.38:0/3323962888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f00c9260 msgr2=0x7f89f014f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f00c9260 0x7f89f014f7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f89e800cd30 tx=0x7f89e8001b00 comp rx=0 tx=0).stop 2024-08-15T17:59:08.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 -- 172.21.15.38:0/3323962888 shutdown_connections 2024-08-15T17:59:08.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89d0060860 0x7f89d0062d10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f0159b90 0x7f89f00c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f0153ba0 0x7f89f0169650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 --2- 172.21.15.38:0/3323962888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f00c9260 0x7f89f014f7a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 -- 172.21.15.38:0/3323962888 >> 172.21.15.38:0/3323962888 conn(0x7f89f00cd9c0 msgr2=0x7f89f006e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:08.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 -- 172.21.15.38:0/3323962888 shutdown_connections 2024-08-15T17:59:08.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.417+0000 7f89f685d700 1 -- 172.21.15.38:0/3323962888 wait complete. 2024-08-15T17:59:08.437 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:59:08.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.545+0000 7fc4354d9700 1 Processor -- start 2024-08-15T17:59:08.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.545+0000 7fc4354d9700 1 -- start start 2024-08-15T17:59:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.545+0000 7fc4354d9700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc4300b2210 0x7fc4300b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.545+0000 7fc4354d9700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4300b5040 0x7fc4300b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.545+0000 7fc4354d9700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4300b5a00 con 0x7fc4300cc840 2024-08-15T17:59:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.545+0000 7fc4354d9700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4300af570 con 0x7fc4300b2210 2024-08-15T17:59:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.545+0000 7fc4354d9700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc430159ce0 con 0x7fc4300b5040 2024-08-15T17:59:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42e7fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc4300b2210 0x7fc4300b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42e7fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc4300b2210 0x7fc4300b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35258/0 (socket says 172.21.15.38:35258) 2024-08-15T17:59:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42e7fc700 1 -- 172.21.15.38:0/2813679250 learned_addr learned my addr 172.21.15.38:0/2813679250 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42f7fe700 1 --2- 172.21.15.38:0/2813679250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4300b5040 0x7fc4300b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42e7fc700 1 -- 172.21.15.38:0/2813679250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4300b5040 msgr2=0x7fc4300b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:08.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42e7fc700 1 --2- 172.21.15.38:0/2813679250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4300b5040 0x7fc4300b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42e7fc700 1 -- 172.21.15.38:0/2813679250 >> v1:172.21.15.104:6789/0 conn(0x7fc4300cc840 legacy=0x7fc4300ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42e7fc700 1 -- 172.21.15.38:0/2813679250 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc430159fc0 con 0x7fc4300b2210 2024-08-15T17:59:08.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42f7fe700 1 --2- 172.21.15.38:0/2813679250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4300b5040 0x7fc4300b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:08.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42e7fc700 1 --2- 172.21.15.38:0/2813679250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4300b2210 0x7fc4300b2670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc4200099b0 tx=0x7fc420009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38639a3344b5762 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:08.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42dffb700 1 -- 172.21.15.38:0/2813679250 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42000b860 con 0x7fc4300b2210 2024-08-15T17:59:08.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42dffb700 1 -- 172.21.15.38:0/2813679250 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42000b9c0 con 0x7fc4300b2210 2024-08-15T17:59:08.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc42dffb700 1 -- 172.21.15.38:0/2813679250 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42000bd60 con 0x7fc4300b2210 2024-08-15T17:59:08.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc4354d9700 1 -- 172.21.15.38:0/2813679250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4300b2210 msgr2=0x7fc4300b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc4354d9700 1 --2- 172.21.15.38:0/2813679250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4300b2210 0x7fc4300b2670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc4200099b0 tx=0x7fc420009250 comp rx=0 tx=0).stop 2024-08-15T17:59:08.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc4354d9700 1 -- 172.21.15.38:0/2813679250 shutdown_connections 2024-08-15T17:59:08.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc4354d9700 1 --2- 172.21.15.38:0/2813679250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4300b5040 0x7fc4300b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc4354d9700 1 --2- 172.21.15.38:0/2813679250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4300b2210 0x7fc4300b2670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc4354d9700 1 -- 172.21.15.38:0/2813679250 >> 172.21.15.38:0/2813679250 conn(0x7fc4300682c0 msgr2=0x7fc4300686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:08.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc4354d9700 1 -- 172.21.15.38:0/2813679250 shutdown_connections 2024-08-15T17:59:08.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.549+0000 7fc4354d9700 1 -- 172.21.15.38:0/2813679250 wait complete. 2024-08-15T17:59:08.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc4354d9700 1 Processor -- start 2024-08-15T17:59:08.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc4354d9700 1 -- start start 2024-08-15T17:59:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc4354d9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4300b2210 0x7fc430152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc4354d9700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4300b5040 0x7fc4300c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc4354d9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4300cc840 0x7fc4300c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc4354d9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc430152d40 con 0x7fc4300b2210 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc4354d9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43015b6a0 con 0x7fc4300cc840 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc4354d9700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43015b980 con 0x7fc4300b5040 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc42effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4300b2210 0x7fc430152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc42effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4300b2210 0x7fc430152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35260/0 (socket says 172.21.15.38:35260) 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc42effd700 1 -- 172.21.15.38:0/1696170236 learned_addr learned my addr 172.21.15.38:0/1696170236 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc42e7fc700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4300b5040 0x7fc4300c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.553+0000 7fc42f7fe700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4300cc840 0x7fc4300c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc42effd700 1 -- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4300b5040 msgr2=0x7fc4300c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc42effd700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4300b5040 0x7fc4300c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc42effd700 1 -- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4300cc840 msgr2=0x7fc4300c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc42effd700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4300cc840 0x7fc4300c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc42effd700 1 -- 172.21.15.38:0/1696170236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4200094f0 con 0x7fc4300b2210 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc42effd700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4300b2210 0x7fc430152800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc41800c590 tx=0x7fc41800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc426ffd700 1 -- 172.21.15.38:0/1696170236 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418011590 con 0x7fc4300b2210 2024-08-15T17:59:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc426ffd700 1 -- 172.21.15.38:0/1696170236 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc418011710 con 0x7fc4300b2210 2024-08-15T17:59:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc426ffd700 1 -- 172.21.15.38:0/1696170236 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418011ad0 con 0x7fc4300b2210 2024-08-15T17:59:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc4354d9700 1 -- 172.21.15.38:0/1696170236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43015bc60 con 0x7fc4300b2210 2024-08-15T17:59:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc4354d9700 1 -- 172.21.15.38:0/1696170236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43015c220 con 0x7fc4300b2210 2024-08-15T17:59:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc4354d9700 1 -- 172.21.15.38:0/1696170236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4300645c0 con 0x7fc4300b2210 2024-08-15T17:59:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc426ffd700 1 -- 172.21.15.38:0/1696170236 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc41801b3b0 con 0x7fc4300b2210 2024-08-15T17:59:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc426ffd700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc410060860 0x7fc410062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc426ffd700 1 -- 172.21.15.38:0/1696170236 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc41807bf70 con 0x7fc4300b2210 2024-08-15T17:59:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.557+0000 7fc42e7fc700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc410060860 0x7fc410062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.561+0000 7fc42e7fc700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc410060860 0x7fc410062d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc4200099b0 tx=0x7fc420017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.569+0000 7fc426ffd700 1 -- 172.21.15.38:0/1696170236 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc41804c3c0 con 0x7fc4300b2210 2024-08-15T17:59:08.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.729+0000 7fc4354d9700 1 -- 172.21.15.38:0/1696170236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4300b1570 con 0x7fc4300b2210 2024-08-15T17:59:08.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.729+0000 7fc426ffd700 1 -- 172.21.15.38:0/1696170236 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc418017020 con 0x7fc4300b2210 2024-08-15T17:59:08.733 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 -- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc410060860 msgr2=0x7fc410062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc410060860 0x7fc410062d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc4200099b0 tx=0x7fc420017000 comp rx=0 tx=0).stop 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 -- 172.21.15.38:0/1696170236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4300b2210 msgr2=0x7fc430152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4300b2210 0x7fc430152800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc41800c590 tx=0x7fc41800c8a0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 -- 172.21.15.38:0/1696170236 shutdown_connections 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fc410060860 0x7fc410062d10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4300cc840 0x7fc4300c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4300b5040 0x7fc4300c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 --2- 172.21.15.38:0/1696170236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4300b2210 0x7fc430152800 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 -- 172.21.15.38:0/1696170236 >> 172.21.15.38:0/1696170236 conn(0x7fc4300682c0 msgr2=0x7fc4300affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 -- 172.21.15.38:0/1696170236 shutdown_connections 2024-08-15T17:59:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.733+0000 7fc4354d9700 1 -- 172.21.15.38:0/1696170236 wait complete. 2024-08-15T17:59:08.747 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:59:08.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f552e383700 1 Processor -- start 2024-08-15T17:59:08.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f552e383700 1 -- start start 2024-08-15T17:59:08.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f552e383700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55280aac50 0x7f552815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f552e383700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f552815a2e0 0x7f55281574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f552e383700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552815b010 con 0x7f55280b3b30 2024-08-15T17:59:08.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f552e383700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552815bc90 con 0x7f552815a2e0 2024-08-15T17:59:08.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f552e383700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552815b2f0 con 0x7f55280aac50 2024-08-15T17:59:08.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f55277fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f552815a2e0 0x7f55281574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f552c920700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f55280b3b30 0x7f55280b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35322/0 (socket says 172.21.15.38:35322) 2024-08-15T17:59:08.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f552c920700 1 -- 172.21.15.38:0/1668038344 learned_addr learned my addr 172.21.15.38:0/1668038344 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:08.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f5527fff700 1 --2- 172.21.15.38:0/1668038344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55280aac50 0x7f552815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f55277fe700 1 -- 172.21.15.38:0/1668038344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55280aac50 msgr2=0x7f552815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:08.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f55277fe700 1 --2- 172.21.15.38:0/1668038344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55280aac50 0x7f552815aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f55277fe700 1 -- 172.21.15.38:0/1668038344 >> v1:172.21.15.38:6789/0 conn(0x7f55280b3b30 legacy=0x7f55280b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f55277fe700 1 -- 172.21.15.38:0/1668038344 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552815b5d0 con 0x7f552815a2e0 2024-08-15T17:59:08.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f5526ffd700 1 -- 172.21.15.38:0/1668038344 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744295080 0 0) 0x7f552815b010 con 0x7f55280b3b30 2024-08-15T17:59:08.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f55277fe700 1 --2- 172.21.15.38:0/1668038344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552815a2e0 0x7f55281574a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f55100098e0 tx=0x7f55100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d7deb6736089eb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:08.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f5526ffd700 1 -- 172.21.15.38:0/1668038344 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551000b640 con 0x7f552815a2e0 2024-08-15T17:59:08.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f5526ffd700 1 -- 172.21.15.38:0/1668038344 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551000b7a0 con 0x7f552815a2e0 2024-08-15T17:59:08.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.849+0000 7f5526ffd700 1 -- 172.21.15.38:0/1668038344 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551000bb40 con 0x7f552815a2e0 2024-08-15T17:59:08.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 -- 172.21.15.38:0/1668038344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552815a2e0 msgr2=0x7f55281574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 --2- 172.21.15.38:0/1668038344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552815a2e0 0x7f55281574a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f55100098e0 tx=0x7f55100091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 -- 172.21.15.38:0/1668038344 shutdown_connections 2024-08-15T17:59:08.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 --2- 172.21.15.38:0/1668038344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552815a2e0 0x7f55281574a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 --2- 172.21.15.38:0/1668038344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55280aac50 0x7f552815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 -- 172.21.15.38:0/1668038344 >> 172.21.15.38:0/1668038344 conn(0x7f55280994d0 msgr2=0x7f55280675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:08.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 -- 172.21.15.38:0/1668038344 shutdown_connections 2024-08-15T17:59:08.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 -- 172.21.15.38:0/1668038344 wait complete. 2024-08-15T17:59:08.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 Processor -- start 2024-08-15T17:59:08.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 -- start start 2024-08-15T17:59:08.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55280aac50 0x7f55281689a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55280b3b30 0x7f552809d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f552815a2e0 0x7f55281425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552809d020 con 0x7f55280aac50 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552816d580 con 0x7f552815a2e0 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f552e383700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552809c410 con 0x7f55280b3b30 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f5527fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55280aac50 0x7f55281689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f5527fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55280aac50 0x7f55281689a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35276/0 (socket says 172.21.15.38:35276) 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f5527fff700 1 -- 172.21.15.38:0/2281813935 learned_addr learned my addr 172.21.15.38:0/2281813935 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.853+0000 7f55277fe700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55280b3b30 0x7f552809d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f552c920700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f552815a2e0 0x7f55281425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f5527fff700 1 -- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55280b3b30 msgr2=0x7f552809d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f5527fff700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55280b3b30 0x7f552809d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f5527fff700 1 -- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f552815a2e0 msgr2=0x7f55281425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:08.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f5527fff700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f552815a2e0 0x7f55281425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:08.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f5527fff700 1 -- 172.21.15.38:0/2281813935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5510009420 con 0x7f55280aac50 2024-08-15T17:59:08.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f5527fff700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55280aac50 0x7f55281689a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5518009f40 tx=0x7f551800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:08.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f551ffff700 1 -- 172.21.15.38:0/2281813935 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800cc20 con 0x7f55280aac50 2024-08-15T17:59:08.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f551ffff700 1 -- 172.21.15.38:0/2281813935 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551800f040 con 0x7f55280aac50 2024-08-15T17:59:08.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f551ffff700 1 -- 172.21.15.38:0/2281813935 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518014480 con 0x7f55280aac50 2024-08-15T17:59:08.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f552e383700 1 -- 172.21.15.38:0/2281813935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552809c6f0 con 0x7f55280aac50 2024-08-15T17:59:08.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f552e383700 1 -- 172.21.15.38:0/2281813935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552809ccb0 con 0x7f55280aac50 2024-08-15T17:59:08.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f552e383700 1 -- 172.21.15.38:0/2281813935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55280645c0 con 0x7f55280aac50 2024-08-15T17:59:08.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f551ffff700 1 -- 172.21.15.38:0/2281813935 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f55180147f0 con 0x7f55280aac50 2024-08-15T17:59:08.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f551ffff700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f550c060890 0x7f550c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:08.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.857+0000 7f551ffff700 1 -- 172.21.15.38:0/2281813935 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f551807d300 con 0x7f55280aac50 2024-08-15T17:59:08.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.861+0000 7f55277fe700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f550c060890 0x7f550c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:08.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.861+0000 7f55277fe700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f550c060890 0x7f550c062d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f55100093f0 tx=0x7f5510017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:08.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:08.865+0000 7f551ffff700 1 -- 172.21.15.38:0/2281813935 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f551804d750 con 0x7f55280aac50 2024-08-15T17:59:09.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.069+0000 7f552e383700 1 -- 172.21.15.38:0/2281813935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5528145590 con 0x7f55280aac50 2024-08-15T17:59:09.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f551ffff700 1 -- 172.21.15.38:0/2281813935 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f5518050d70 con 0x7f55280aac50 2024-08-15T17:59:09.075 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:09.075 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2444719791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2444719791},{"type":"v1","addr":"172.21.15.38:6833","nonce":2444719791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2667173752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2667173752},{"type":"v1","addr":"172.21.15.38:6835","nonce":2667173752}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1185013072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1185013072},{"type":"v1","addr":"172.21.15.104:6835","nonce":1185013072}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:58:35.622492+0000","modified":"2024-08-15T17:58:38.885045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/3628618356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3628618356},{"type":"v1","addr":"172.21.15.104:6837","nonce":3628618356}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:59:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 -- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f550c060890 msgr2=0x7f550c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f550c060890 0x7f550c062d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f55100093f0 tx=0x7f5510017000 comp rx=0 tx=0).stop 2024-08-15T17:59:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 -- 172.21.15.38:0/2281813935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55280aac50 msgr2=0x7f55281689a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55280aac50 0x7f55281689a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5518009f40 tx=0x7f551800b040 comp rx=0 tx=0).stop 2024-08-15T17:59:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 -- 172.21.15.38:0/2281813935 shutdown_connections 2024-08-15T17:59:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f550c060890 0x7f550c062d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f552815a2e0 0x7f55281425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55280b3b30 0x7f552809d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 --2- 172.21.15.38:0/2281813935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55280aac50 0x7f55281689a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 -- 172.21.15.38:0/2281813935 >> 172.21.15.38:0/2281813935 conn(0x7f55280994d0 msgr2=0x7f55281598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 -- 172.21.15.38:0/2281813935 shutdown_connections 2024-08-15T17:59:09.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.073+0000 7f552e383700 1 -- 172.21.15.38:0/2281813935 wait complete. 2024-08-15T17:59:09.079 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-08-15T17:59:09.093 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:59:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.197+0000 7f2f4dd48700 1 Processor -- start 2024-08-15T17:59:09.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.197+0000 7f2f4dd48700 1 -- start start 2024-08-15T17:59:09.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.197+0000 7f2f4dd48700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2f480c6c80 0x7f2f480af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f4dd48700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480c7dc0 con 0x7f2f48159610 2024-08-15T17:59:09.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f4dd48700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480c71b0 con 0x7f2f48159240 2024-08-15T17:59:09.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f4dd48700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4806c170 con 0x7f2f480c6c80 2024-08-15T17:59:09.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f46ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2f480c6c80 0x7f2f480af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f46ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2f480c6c80 0x7f2f480af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35280/0 (socket says 172.21.15.38:35280) 2024-08-15T17:59:09.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f46ffd700 1 -- 172.21.15.38:0/1672391868 learned_addr learned my addr 172.21.15.38:0/1672391868 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f46ffd700 1 -- 172.21.15.38:0/1672391868 >> v1:172.21.15.104:6789/0 conn(0x7f2f48159240 legacy=0x7f2f4806ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f46ffd700 1 -- 172.21.15.38:0/1672391868 >> v1:172.21.15.38:6789/0 conn(0x7f2f48159610 legacy=0x7f2f480686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f46ffd700 1 -- 172.21.15.38:0/1672391868 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480c7490 con 0x7f2f480c6c80 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f46ffd700 1 --2- 172.21.15.38:0/1672391868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f480c6c80 0x7f2f480af5d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0099b0 tx=0x7f2f3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70254db581c46fc server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f467fc700 1 -- 172.21.15.38:0/1672391868 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00b860 con 0x7f2f480c6c80 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f467fc700 1 -- 172.21.15.38:0/1672391868 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c00b9c0 con 0x7f2f480c6c80 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f467fc700 1 -- 172.21.15.38:0/1672391868 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00bd60 con 0x7f2f480c6c80 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f4dd48700 1 -- 172.21.15.38:0/1672391868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f480c6c80 msgr2=0x7f2f480af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f4dd48700 1 --2- 172.21.15.38:0/1672391868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f480c6c80 0x7f2f480af5d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0099b0 tx=0x7f2f3c009250 comp rx=0 tx=0).stop 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f4dd48700 1 -- 172.21.15.38:0/1672391868 shutdown_connections 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f4dd48700 1 --2- 172.21.15.38:0/1672391868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f480c6c80 0x7f2f480af5d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f4dd48700 1 -- 172.21.15.38:0/1672391868 >> 172.21.15.38:0/1672391868 conn(0x7f2f480c9c20 msgr2=0x7f2f4806d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:09.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f4dd48700 1 -- 172.21.15.38:0/1672391868 shutdown_connections 2024-08-15T17:59:09.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f4dd48700 1 -- 172.21.15.38:0/1672391868 wait complete. 2024-08-15T17:59:09.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.201+0000 7f2f4dd48700 1 Processor -- start 2024-08-15T17:59:09.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f4dd48700 1 -- start start 2024-08-15T17:59:09.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f4dd48700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f480c6c80 0x7f2f48146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f4dd48700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f48159240 0x7f2f48067a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f4dd48700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f48159610 0x7f2f48067fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f4dd48700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480a94c0 con 0x7f2f480c6c80 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f4dd48700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480a88b0 con 0x7f2f48159240 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f4dd48700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480a8b90 con 0x7f2f48159610 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f477fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f480c6c80 0x7f2f48146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f477fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f480c6c80 0x7f2f48146e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35282/0 (socket says 172.21.15.38:35282) 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f477fe700 1 -- 172.21.15.38:0/4176501223 learned_addr learned my addr 172.21.15.38:0/4176501223 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f46ffd700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f48159240 0x7f2f48067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f47fff700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f48159610 0x7f2f48067fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f477fe700 1 -- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f48159610 msgr2=0x7f2f48067fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f477fe700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f48159610 0x7f2f48067fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f477fe700 1 -- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f48159240 msgr2=0x7f2f48067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f477fe700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f48159240 0x7f2f48067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f477fe700 1 -- 172.21.15.38:0/4176501223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0094f0 con 0x7f2f480c6c80 2024-08-15T17:59:09.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f47fff700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f48159610 0x7f2f48067fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:59:09.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f477fe700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f480c6c80 0x7f2f48146e50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800c590 tx=0x7f2f38007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:09.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f2f7fe700 1 -- 172.21.15.38:0/4176501223 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38011590 con 0x7f2f480c6c80 2024-08-15T17:59:09.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f2f7fe700 1 -- 172.21.15.38:0/4176501223 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f38011730 con 0x7f2f480c6c80 2024-08-15T17:59:09.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f2f7fe700 1 -- 172.21.15.38:0/4176501223 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38011af0 con 0x7f2f480c6c80 2024-08-15T17:59:09.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f4dd48700 1 -- 172.21.15.38:0/4176501223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f480a8e70 con 0x7f2f480c6c80 2024-08-15T17:59:09.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.205+0000 7f2f4dd48700 1 -- 172.21.15.38:0/4176501223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f48147390 con 0x7f2f480c6c80 2024-08-15T17:59:09.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.209+0000 7f2f4dd48700 1 -- 172.21.15.38:0/4176501223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f480645c0 con 0x7f2f480c6c80 2024-08-15T17:59:09.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.209+0000 7f2f2f7fe700 1 -- 172.21.15.38:0/4176501223 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2f38012420 con 0x7f2f480c6c80 2024-08-15T17:59:09.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.209+0000 7f2f2f7fe700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f34060860 0x7f2f34062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.209+0000 7f2f2f7fe700 1 -- 172.21.15.38:0/4176501223 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f3807bf40 con 0x7f2f480c6c80 2024-08-15T17:59:09.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.209+0000 7f2f46ffd700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f34060860 0x7f2f34062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.209+0000 7f2f46ffd700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f34060860 0x7f2f34062d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f480c70e0 tx=0x7f2f3c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:09.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.217+0000 7f2f2f7fe700 1 -- 172.21.15.38:0/4176501223 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2f3804c440 con 0x7f2f480c6c80 2024-08-15T17:59:09.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.381+0000 7f2f4dd48700 1 -- 172.21.15.38:0/4176501223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f480cb9f0 con 0x7f2f480c6c80 2024-08-15T17:59:09.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f2f7fe700 1 -- 172.21.15.38:0/4176501223 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f2f38017070 con 0x7f2f480c6c80 2024-08-15T17:59:09.388 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:09.388 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:59:09.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 -- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f34060860 msgr2=0x7f2f34062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f34060860 0x7f2f34062d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f480c70e0 tx=0x7f2f3c017000 comp rx=0 tx=0).stop 2024-08-15T17:59:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 -- 172.21.15.38:0/4176501223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f480c6c80 msgr2=0x7f2f48146e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f480c6c80 0x7f2f48146e50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800c590 tx=0x7f2f38007f30 comp rx=0 tx=0).stop 2024-08-15T17:59:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 -- 172.21.15.38:0/4176501223 shutdown_connections 2024-08-15T17:59:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f34060860 0x7f2f34062d10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f48159610 0x7f2f48067fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f48159240 0x7f2f48067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 --2- 172.21.15.38:0/4176501223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f480c6c80 0x7f2f48146e50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 -- 172.21.15.38:0/4176501223 >> 172.21.15.38:0/4176501223 conn(0x7f2f480c9c20 msgr2=0x7f2f4806ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 -- 172.21.15.38:0/4176501223 shutdown_connections 2024-08-15T17:59:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.385+0000 7f2f4dd48700 1 -- 172.21.15.38:0/4176501223 wait complete. 2024-08-15T17:59:09.403 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-15T17:59:09.403 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:59:09.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.505+0000 7f2b8dd53700 1 Processor -- start 2024-08-15T17:59:09.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.505+0000 7f2b8dd53700 1 -- start start 2024-08-15T17:59:09.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2b88159040 0x7f2b880cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2b880d0010 0x7f2b880d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b880cf190 con 0x7f2b8815cea0 2024-08-15T17:59:09.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8815d9f0 con 0x7f2b880d0010 2024-08-15T17:59:09.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b880d09a0 con 0x7f2b88159040 2024-08-15T17:59:09.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b87fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2b880d0010 0x7f2b880d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b87fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2b880d0010 0x7f2b880d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35290/0 (socket says 172.21.15.38:35290) 2024-08-15T17:59:09.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b87fff700 1 -- 172.21.15.38:0/3171313473 learned_addr learned my addr 172.21.15.38:0/3171313473 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:09.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8cd51700 1 --2- 172.21.15.38:0/3171313473 >> v2:172.21.15.104:3300/0 conn(0x7f2b88159040 0x7f2b880cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b87fff700 1 -- 172.21.15.38:0/3171313473 >> v2:172.21.15.104:3300/0 conn(0x7f2b88159040 msgr2=0x7f2b880cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:09.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b87fff700 1 --2- 172.21.15.38:0/3171313473 >> v2:172.21.15.104:3300/0 conn(0x7f2b88159040 0x7f2b880cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b87fff700 1 -- 172.21.15.38:0/3171313473 >> v1:172.21.15.38:6789/0 conn(0x7f2b8815cea0 legacy=0x7f2b8815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b87fff700 1 -- 172.21.15.38:0/3171313473 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b880ce5e0 con 0x7f2b880d0010 2024-08-15T17:59:09.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b87fff700 1 --2- 172.21.15.38:0/3171313473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b880d0010 0x7f2b880d0460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2b88070d70 tx=0x7f2b78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e06f5a73db8195f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:09.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b877fe700 1 -- 172.21.15.38:0/3171313473 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78009e70 con 0x7f2b880d0010 2024-08-15T17:59:09.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b877fe700 1 -- 172.21.15.38:0/3171313473 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7800b860 con 0x7f2b880d0010 2024-08-15T17:59:09.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b877fe700 1 -- 172.21.15.38:0/3171313473 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7800bc00 con 0x7f2b880d0010 2024-08-15T17:59:09.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 -- 172.21.15.38:0/3171313473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b880d0010 msgr2=0x7f2b880d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 --2- 172.21.15.38:0/3171313473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b880d0010 0x7f2b880d0460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2b88070d70 tx=0x7f2b78009250 comp rx=0 tx=0).stop 2024-08-15T17:59:09.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 -- 172.21.15.38:0/3171313473 shutdown_connections 2024-08-15T17:59:09.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 --2- 172.21.15.38:0/3171313473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b880d0010 0x7f2b880d0460 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 --2- 172.21.15.38:0/3171313473 >> v2:172.21.15.104:3300/0 conn(0x7f2b88159040 0x7f2b880cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 -- 172.21.15.38:0/3171313473 >> 172.21.15.38:0/3171313473 conn(0x7f2b88068250 msgr2=0x7f2b88068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:09.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 -- 172.21.15.38:0/3171313473 shutdown_connections 2024-08-15T17:59:09.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.509+0000 7f2b8dd53700 1 -- 172.21.15.38:0/3171313473 wait complete. 2024-08-15T17:59:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.513+0000 7f2b8dd53700 1 Processor -- start 2024-08-15T17:59:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.513+0000 7f2b8dd53700 1 -- start start 2024-08-15T17:59:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8dd53700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b880d0010 0x7f2b880b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8dd53700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b88159040 0x7f2b880b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8dd53700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8815cea0 0x7f2b88148540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8dd53700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b88169790 con 0x7f2b88159040 2024-08-15T17:59:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8dd53700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b880b0b10 con 0x7f2b8815cea0 2024-08-15T17:59:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8dd53700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b880aff00 con 0x7f2b880d0010 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b87fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b88159040 0x7f2b880b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b87fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b88159040 0x7f2b880b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35294/0 (socket says 172.21.15.38:35294) 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b87fff700 1 -- 172.21.15.38:0/603010925 learned_addr learned my addr 172.21.15.38:0/603010925 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8cd51700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b880d0010 0x7f2b880b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8d552700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8815cea0 0x7f2b88148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b87fff700 1 -- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b880d0010 msgr2=0x7f2b880b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b87fff700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b880d0010 0x7f2b880b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b87fff700 1 -- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8815cea0 msgr2=0x7f2b88148540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b87fff700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8815cea0 0x7f2b88148540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b87fff700 1 -- 172.21.15.38:0/603010925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b780094f0 con 0x7f2b88159040 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8cd51700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b880d0010 0x7f2b880b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8d552700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8815cea0 0x7f2b88148540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:09.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b87fff700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b88159040 0x7f2b880b5710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2b88088ef0 tx=0x7f2b78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:09.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b84ff9700 1 -- 172.21.15.38:0/603010925 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7800bbe0 con 0x7f2b88159040 2024-08-15T17:59:09.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b84ff9700 1 -- 172.21.15.38:0/603010925 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78011420 con 0x7f2b88159040 2024-08-15T17:59:09.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8dd53700 1 -- 172.21.15.38:0/603010925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b880b01e0 con 0x7f2b88159040 2024-08-15T17:59:09.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b84ff9700 1 -- 172.21.15.38:0/603010925 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b780117c0 con 0x7f2b88159040 2024-08-15T17:59:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.517+0000 7f2b8dd53700 1 -- 172.21.15.38:0/603010925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b880b07a0 con 0x7f2b88159040 2024-08-15T17:59:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.521+0000 7f2b8dd53700 1 -- 172.21.15.38:0/603010925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b880b0ca0 con 0x7f2b88159040 2024-08-15T17:59:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.521+0000 7f2b84ff9700 1 -- 172.21.15.38:0/603010925 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2b78011bb0 con 0x7f2b88159040 2024-08-15T17:59:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.521+0000 7f2b84ff9700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2b68060860 0x7f2b68062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.521+0000 7f2b84ff9700 1 -- 172.21.15.38:0/603010925 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2b7807df10 con 0x7f2b88159040 2024-08-15T17:59:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.521+0000 7f2b8cd51700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2b68060860 0x7f2b68062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.521+0000 7f2b8cd51700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2b68060860 0x7f2b68062d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b88149920 tx=0x7f2b700091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:09.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.529+0000 7f2b84ff9700 1 -- 172.21.15.38:0/603010925 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2b7804e360 con 0x7f2b88159040 2024-08-15T17:59:09.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.729+0000 7f2b8dd53700 1 -- 172.21.15.38:0/603010925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b880643a0 con 0x7f2b88159040 2024-08-15T17:59:09.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.729+0000 7f2b84ff9700 1 -- 172.21.15.38:0/603010925 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f2b78011e60 con 0x7f2b88159040 2024-08-15T17:59:09.731 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:09.732 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2444719791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2444719791},{"type":"v1","addr":"172.21.15.38:6833","nonce":2444719791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2667173752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2667173752},{"type":"v1","addr":"172.21.15.38:6835","nonce":2667173752}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1185013072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1185013072},{"type":"v1","addr":"172.21.15.104:6835","nonce":1185013072}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:58:35.622492+0000","modified":"2024-08-15T17:58:38.885045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/3628618356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3628618356},{"type":"v1","addr":"172.21.15.104:6837","nonce":3628618356}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:59:09.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 -- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2b68060860 msgr2=0x7f2b68062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2b68060860 0x7f2b68062d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b88149920 tx=0x7f2b700091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 -- 172.21.15.38:0/603010925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b88159040 msgr2=0x7f2b880b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b88159040 0x7f2b880b5710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2b88088ef0 tx=0x7f2b78017000 comp rx=0 tx=0).stop 2024-08-15T17:59:09.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 -- 172.21.15.38:0/603010925 shutdown_connections 2024-08-15T17:59:09.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2b68060860 0x7f2b68062d10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8815cea0 0x7f2b88148540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b88159040 0x7f2b880b5710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 --2- 172.21.15.38:0/603010925 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b880d0010 0x7f2b880b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 -- 172.21.15.38:0/603010925 >> 172.21.15.38:0/603010925 conn(0x7f2b88068250 msgr2=0x7f2b880b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:09.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 -- 172.21.15.38:0/603010925 shutdown_connections 2024-08-15T17:59:09.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.733+0000 7f2b8dd53700 1 -- 172.21.15.38:0/603010925 wait complete. 2024-08-15T17:59:09.737 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-08-15T17:59:09.750 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:59:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.853+0000 7f6b7ffff700 1 Processor -- start 2024-08-15T17:59:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.853+0000 7f6b7ffff700 1 -- start start 2024-08-15T17:59:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.853+0000 7f6b7ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6b800ad1b0 0x7f6b800b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.853+0000 7f6b7ffff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b800c86e0 con 0x7f6b8015c620 2024-08-15T17:59:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.853+0000 7f6b7ffff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8015d210 con 0x7f6b8015cd90 2024-08-15T17:59:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.853+0000 7f6b7ffff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b800cb850 con 0x7f6b800ad1b0 2024-08-15T17:59:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7effd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6b8015c620 0x7f6b800acaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35350/0 (socket says 172.21.15.38:35350) 2024-08-15T17:59:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7effd700 1 -- 172.21.15.38:0/2618106408 learned_addr learned my addr 172.21.15.38:0/2618106408 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7e7fc700 1 --2- 172.21.15.38:0/2618106408 >> v2:172.21.15.104:3300/0 conn(0x7f6b800ad1b0 0x7f6b800b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7dffb700 1 -- 172.21.15.38:0/2618106408 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451569922 0 0) 0x7f6b800c86e0 con 0x7f6b8015c620 2024-08-15T17:59:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7dffb700 1 -- 172.21.15.38:0/2618106408 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b68003570 con 0x7f6b8015c620 2024-08-15T17:59:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7dffb700 1 -- 172.21.15.38:0/2618106408 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691813765 0 0) 0x7f6b8015d210 con 0x7f6b8015cd90 2024-08-15T17:59:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7dffb700 1 -- 172.21.15.38:0/2618106408 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b800c86e0 con 0x7f6b8015cd90 2024-08-15T17:59:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7e7fc700 1 -- 172.21.15.38:0/2618106408 >> v1:172.21.15.104:6789/0 conn(0x7f6b8015cd90 legacy=0x7f6b800c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7e7fc700 1 -- 172.21.15.38:0/2618106408 >> v1:172.21.15.38:6789/0 conn(0x7f6b8015c620 legacy=0x7f6b800acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7e7fc700 1 -- 172.21.15.38:0/2618106408 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8015d490 con 0x7f6b800ad1b0 2024-08-15T17:59:09.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7e7fc700 1 --2- 172.21.15.38:0/2618106408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b800ad1b0 0x7f6b800b0320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b740098e0 tx=0x7f6b740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9e67ba36de7d7c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:09.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7dffb700 1 -- 172.21.15.38:0/2618106408 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b74011630 con 0x7f6b800ad1b0 2024-08-15T17:59:09.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7dffb700 1 -- 172.21.15.38:0/2618106408 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b74011790 con 0x7f6b800ad1b0 2024-08-15T17:59:09.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7dffb700 1 -- 172.21.15.38:0/2618106408 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b74011b30 con 0x7f6b800ad1b0 2024-08-15T17:59:09.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7ffff700 1 -- 172.21.15.38:0/2618106408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b800ad1b0 msgr2=0x7f6b800b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7ffff700 1 --2- 172.21.15.38:0/2618106408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b800ad1b0 0x7f6b800b0320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b740098e0 tx=0x7f6b740091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7ffff700 1 -- 172.21.15.38:0/2618106408 shutdown_connections 2024-08-15T17:59:09.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7ffff700 1 --2- 172.21.15.38:0/2618106408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b800ad1b0 0x7f6b800b0320 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7ffff700 1 -- 172.21.15.38:0/2618106408 >> 172.21.15.38:0/2618106408 conn(0x7f6b800ca900 msgr2=0x7f6b800cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:09.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7ffff700 1 -- 172.21.15.38:0/2618106408 shutdown_connections 2024-08-15T17:59:09.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.857+0000 7f6b7ffff700 1 -- 172.21.15.38:0/2618106408 wait complete. 2024-08-15T17:59:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7ffff700 1 Processor -- start 2024-08-15T17:59:09.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7ffff700 1 -- start start 2024-08-15T17:59:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b800ad1b0 0x7f6b8015a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b8015c620 0x7f6b800c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b8015cd90 0x7f6b800cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7ffff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800c72e0 con 0x7f6b8015cd90 2024-08-15T17:59:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7ffff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8006aa80 con 0x7f6b800ad1b0 2024-08-15T17:59:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7ffff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8006a750 con 0x7f6b8015c620 2024-08-15T17:59:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b8015cd90 0x7f6b800cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b800ad1b0 0x7f6b8015a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b800ad1b0 0x7f6b8015a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:52660/0 (socket says 172.21.15.38:52660) 2024-08-15T17:59:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7effd700 1 -- 172.21.15.38:0/862575240 learned_addr learned my addr 172.21.15.38:0/862575240 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:09.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7e7fc700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b8015c620 0x7f6b800c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7effd700 1 -- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b8015c620 msgr2=0x7f6b800c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7effd700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b8015c620 0x7f6b800c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7effd700 1 -- 172.21.15.38:0/862575240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b8015cd90 msgr2=0x7f6b800cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7effd700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b8015cd90 0x7f6b800cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7effd700 1 -- 172.21.15.38:0/862575240 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b74009420 con 0x7f6b800ad1b0 2024-08-15T17:59:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7f7fe700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b8015cd90 0x7f6b800cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7effd700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b800ad1b0 0x7f6b8015a150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800cd80 tx=0x7f6b78001b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b66ffd700 1 -- 172.21.15.38:0/862575240 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7800aa80 con 0x7f6b800ad1b0 2024-08-15T17:59:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b66ffd700 1 -- 172.21.15.38:0/862575240 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7800abe0 con 0x7f6b800ad1b0 2024-08-15T17:59:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b66ffd700 1 -- 172.21.15.38:0/862575240 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b780070f0 con 0x7f6b800ad1b0 2024-08-15T17:59:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7ffff700 1 -- 172.21.15.38:0/862575240 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b800c6a40 con 0x7f6b800ad1b0 2024-08-15T17:59:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.861+0000 7f6b7ffff700 1 -- 172.21.15.38:0/862575240 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b800c7080 con 0x7f6b800ad1b0 2024-08-15T17:59:09.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.865+0000 7f6b7ffff700 1 -- 172.21.15.38:0/862575240 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8015a7b0 con 0x7f6b800ad1b0 2024-08-15T17:59:09.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.865+0000 7f6b66ffd700 1 -- 172.21.15.38:0/862575240 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6b780074e0 con 0x7f6b800ad1b0 2024-08-15T17:59:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.865+0000 7f6b66ffd700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b6c060860 0x7f6b6c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.865+0000 7f6b66ffd700 1 -- 172.21.15.38:0/862575240 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6b7807cf20 con 0x7f6b800ad1b0 2024-08-15T17:59:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.865+0000 7f6b7e7fc700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b6c060860 0x7f6b6c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.865+0000 7f6b7e7fc700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b6c060860 0x7f6b6c062d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6b740098e0 tx=0x7f6b74017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:09.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:09.877+0000 7f6b66ffd700 1 -- 172.21.15.38:0/862575240 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6b7804d3f0 con 0x7f6b800ad1b0 2024-08-15T17:59:10.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.081+0000 7f6b7ffff700 1 -- 172.21.15.38:0/862575240 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b800643a0 con 0x7f6b800ad1b0 2024-08-15T17:59:10.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.081+0000 7f6b66ffd700 1 -- 172.21.15.38:0/862575240 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f6b78050a10 con 0x7f6b800ad1b0 2024-08-15T17:59:10.085 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:10.085 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2444719791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2444719791},{"type":"v1","addr":"172.21.15.38:6833","nonce":2444719791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2667173752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":2667173752},{"type":"v1","addr":"172.21.15.38:6835","nonce":2667173752}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1185013072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1185013072},{"type":"v1","addr":"172.21.15.104:6835","nonce":1185013072}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:58:35.622492+0000","modified":"2024-08-15T17:58:38.885045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/3628618356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3628618356},{"type":"v1","addr":"172.21.15.104:6837","nonce":3628618356}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:59:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.081+0000 7f6b7ffff700 1 -- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b6c060860 msgr2=0x7f6b6c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.081+0000 7f6b7ffff700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b6c060860 0x7f6b6c062d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6b740098e0 tx=0x7f6b74017000 comp rx=0 tx=0).stop 2024-08-15T17:59:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.081+0000 7f6b7ffff700 1 -- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b800ad1b0 msgr2=0x7f6b8015a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.081+0000 7f6b7ffff700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b800ad1b0 0x7f6b8015a150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800cd80 tx=0x7f6b78001b90 comp rx=0 tx=0).stop 2024-08-15T17:59:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.081+0000 7f6b7ffff700 1 -- 172.21.15.38:0/862575240 shutdown_connections 2024-08-15T17:59:10.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.085+0000 7f6b7ffff700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b6c060860 0x7f6b6c062d10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.085+0000 7f6b7ffff700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b8015cd90 0x7f6b800cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.085+0000 7f6b7ffff700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b8015c620 0x7f6b800c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.085+0000 7f6b7ffff700 1 --2- 172.21.15.38:0/862575240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b800ad1b0 0x7f6b8015a150 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.085+0000 7f6b7ffff700 1 -- 172.21.15.38:0/862575240 >> 172.21.15.38:0/862575240 conn(0x7f6b800ca900 msgr2=0x7f6b800acaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:10.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.085+0000 7f6b7ffff700 1 -- 172.21.15.38:0/862575240 shutdown_connections 2024-08-15T17:59:10.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.085+0000 7f6b7ffff700 1 -- 172.21.15.38:0/862575240 wait complete. 2024-08-15T17:59:10.090 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-08-15T17:59:10.100 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:59:10.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.201+0000 7f3dec59e700 1 Processor -- start 2024-08-15T17:59:10.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.201+0000 7f3dec59e700 1 -- start start 2024-08-15T17:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3de4142890 0x7f3de40cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3de4142c60 0x7f3de40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3de415cea0 0x7f3de415d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de415d880 con 0x7f3de4142c60 2024-08-15T17:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de40cc800 con 0x7f3de415cea0 2024-08-15T17:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40cbbf0 con 0x7f3de4142890 2024-08-15T17:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9b39700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3de4142c60 0x7f3de40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9b39700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3de4142c60 0x7f3de40cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35316/0 (socket says 172.21.15.38:35316) 2024-08-15T17:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9b39700 1 -- 172.21.15.38:0/2357842405 learned_addr learned my addr 172.21.15.38:0/2357842405 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dea33a700 1 --2- 172.21.15.38:0/2357842405 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3de4142890 0x7f3de40cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3deab3b700 1 --2- 172.21.15.38:0/2357842405 >> v2:172.21.15.104:3300/0 conn(0x7f3de415cea0 0x7f3de415d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9b39700 1 -- 172.21.15.38:0/2357842405 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3de4142890 msgr2=0x7f3de40cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9b39700 1 --2- 172.21.15.38:0/2357842405 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3de4142890 0x7f3de40cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9b39700 1 -- 172.21.15.38:0/2357842405 >> v2:172.21.15.104:3300/0 conn(0x7f3de415cea0 msgr2=0x7f3de415d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9b39700 1 --2- 172.21.15.38:0/2357842405 >> v2:172.21.15.104:3300/0 conn(0x7f3de415cea0 0x7f3de415d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9b39700 1 -- 172.21.15.38:0/2357842405 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40cbed0 con 0x7f3de4142c60 2024-08-15T17:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9b39700 1 --2- 172.21.15.38:0/2357842405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de4142c60 0x7f3de40cb5e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3de41430b0 tx=0x7f3dd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96951a19d37887d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:10.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9338700 1 -- 172.21.15.38:0/2357842405 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400b640 con 0x7f3de4142c60 2024-08-15T17:59:10.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9338700 1 -- 172.21.15.38:0/2357842405 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd400b7a0 con 0x7f3de4142c60 2024-08-15T17:59:10.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3de9338700 1 -- 172.21.15.38:0/2357842405 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400bb60 con 0x7f3de4142c60 2024-08-15T17:59:10.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 -- 172.21.15.38:0/2357842405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de4142c60 msgr2=0x7f3de40cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 --2- 172.21.15.38:0/2357842405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de4142c60 0x7f3de40cb5e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3de41430b0 tx=0x7f3dd4009250 comp rx=0 tx=0).stop 2024-08-15T17:59:10.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 -- 172.21.15.38:0/2357842405 shutdown_connections 2024-08-15T17:59:10.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 --2- 172.21.15.38:0/2357842405 >> v2:172.21.15.104:3300/0 conn(0x7f3de415cea0 0x7f3de415d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 --2- 172.21.15.38:0/2357842405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de4142c60 0x7f3de40cb5e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 --2- 172.21.15.38:0/2357842405 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3de4142890 0x7f3de40cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 -- 172.21.15.38:0/2357842405 >> 172.21.15.38:0/2357842405 conn(0x7f3de40a9db0 msgr2=0x7f3de40ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 -- 172.21.15.38:0/2357842405 shutdown_connections 2024-08-15T17:59:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.205+0000 7f3dec59e700 1 -- 172.21.15.38:0/2357842405 wait complete. 2024-08-15T17:59:10.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.209+0000 7f3dec59e700 1 Processor -- start 2024-08-15T17:59:10.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.209+0000 7f3dec59e700 1 -- start start 2024-08-15T17:59:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.209+0000 7f3dec59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3de4142890 0x7f3de4158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.209+0000 7f3dec59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3de4142c60 0x7f3de413c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3dec59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de415cea0 0x7f3de41375f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3dec59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de413c430 con 0x7f3de415cea0 2024-08-15T17:59:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3dec59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de4159160 con 0x7f3de4142890 2024-08-15T17:59:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3dec59e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de413b820 con 0x7f3de4142c60 2024-08-15T17:59:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3deab3b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de415cea0 0x7f3de41375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3de9b39700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3de4142c60 0x7f3de413c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3deab3b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de415cea0 0x7f3de41375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35322/0 (socket says 172.21.15.38:35322) 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3dea33a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3de4142890 0x7f3de4158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3deab3b700 1 -- 172.21.15.38:0/3299957629 learned_addr learned my addr 172.21.15.38:0/3299957629 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3de9b39700 1 -- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3de4142890 msgr2=0x7f3de4158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3de9b39700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3de4142890 0x7f3de4158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3de9b39700 1 -- 172.21.15.38:0/3299957629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de415cea0 msgr2=0x7f3de41375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3de9b39700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de415cea0 0x7f3de41375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3de9b39700 1 -- 172.21.15.38:0/3299957629 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd40094f0 con 0x7f3de4142c60 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3de9b39700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3de4142c60 0x7f3de413c8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3de4064020 tx=0x7f3dd4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3dda7fc700 1 -- 172.21.15.38:0/3299957629 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400b910 con 0x7f3de4142c60 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3dda7fc700 1 -- 172.21.15.38:0/3299957629 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd400ba70 con 0x7f3de4142c60 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3dec59e700 1 -- 172.21.15.38:0/3299957629 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de413bb00 con 0x7f3de4142c60 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3dda7fc700 1 -- 172.21.15.38:0/3299957629 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4011650 con 0x7f3de4142c60 2024-08-15T17:59:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3dec59e700 1 -- 172.21.15.38:0/3299957629 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de413c0c0 con 0x7f3de4142c60 2024-08-15T17:59:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.213+0000 7f3dec59e700 1 -- 172.21.15.38:0/3299957629 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de40645c0 con 0x7f3de4142c60 2024-08-15T17:59:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.217+0000 7f3dda7fc700 1 -- 172.21.15.38:0/3299957629 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3dd4011a00 con 0x7f3de4142c60 2024-08-15T17:59:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.217+0000 7f3dda7fc700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3dc8060860 0x7f3dc8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.217+0000 7f3dda7fc700 1 -- 172.21.15.38:0/3299957629 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3dd407e600 con 0x7f3de4142c60 2024-08-15T17:59:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.217+0000 7f3dea33a700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3dc8060860 0x7f3dc8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.217+0000 7f3dea33a700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3dc8060860 0x7f3dc8062d10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009d10 tx=0x7f3de00091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:10.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.225+0000 7f3dda7fc700 1 -- 172.21.15.38:0/3299957629 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3dd404eab0 con 0x7f3de4142c60 2024-08-15T17:59:10.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.393+0000 7f3dec59e700 1 -- 172.21.15.38:0/3299957629 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3de40aff80 con 0x7f3de4142c60 2024-08-15T17:59:10.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.397+0000 7f3dda7fc700 1 -- 172.21.15.38:0/3299957629 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f3dd4011cb0 con 0x7f3de4142c60 2024-08-15T17:59:10.400 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:10.401 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:58:37.274669+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:58:34.079267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:58:34.621969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:59:10.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 -- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3dc8060860 msgr2=0x7f3dc8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3dc8060860 0x7f3dc8062d10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009d10 tx=0x7f3de00091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 -- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3de4142c60 msgr2=0x7f3de413c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3de4142c60 0x7f3de413c8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3de4064020 tx=0x7f3dd4017000 comp rx=0 tx=0).stop 2024-08-15T17:59:10.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 -- 172.21.15.38:0/3299957629 shutdown_connections 2024-08-15T17:59:10.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3dc8060860 0x7f3dc8062d10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3de415cea0 0x7f3de41375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3de4142c60 0x7f3de413c8c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 --2- 172.21.15.38:0/3299957629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3de4142890 0x7f3de4158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 -- 172.21.15.38:0/3299957629 >> 172.21.15.38:0/3299957629 conn(0x7f3de40a9db0 msgr2=0x7f3de40aca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:10.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 -- 172.21.15.38:0/3299957629 shutdown_connections 2024-08-15T17:59:10.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.401+0000 7f3dec59e700 1 -- 172.21.15.38:0/3299957629 wait complete. 2024-08-15T17:59:10.421 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-08-15T17:59:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e34874700 1 Processor -- start 2024-08-15T17:59:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e34874700 1 -- start start 2024-08-15T17:59:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e34874700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e30158eb0 0x7f3e300aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e34874700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3e300aecc0 0x7f3e3006df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e34874700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3e30159280 0x7f3e3006e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e34874700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e300ac000 con 0x7f3e300aecc0 2024-08-15T17:59:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e34874700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e300ab3f0 con 0x7f3e30159280 2024-08-15T17:59:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e34874700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e300ab6d0 con 0x7f3e30158eb0 2024-08-15T17:59:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e2e59c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3e300aecc0 0x7f3e3006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e2e59c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3e300aecc0 0x7f3e3006df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35330/0 (socket says 172.21.15.38:35330) 2024-08-15T17:59:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e2e59c700 1 -- 172.21.15.38:0/201757232 learned_addr learned my addr 172.21.15.38:0/201757232 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e2f59e700 1 --2- 172.21.15.38:0/201757232 >> v2:172.21.15.104:3300/0 conn(0x7f3e30159280 0x7f3e3006e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e2ed9d700 1 --2- 172.21.15.38:0/201757232 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e30158eb0 0x7f3e300aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e2e59c700 1 -- 172.21.15.38:0/201757232 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e30158eb0 msgr2=0x7f3e300aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:10.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e2e59c700 1 --2- 172.21.15.38:0/201757232 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e30158eb0 0x7f3e300aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e2e59c700 1 -- 172.21.15.38:0/201757232 >> v2:172.21.15.104:3300/0 conn(0x7f3e30159280 msgr2=0x7f3e3006e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:10.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e2e59c700 1 --2- 172.21.15.38:0/201757232 >> v2:172.21.15.104:3300/0 conn(0x7f3e30159280 0x7f3e3006e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.529+0000 7f3e2e59c700 1 -- 172.21.15.38:0/201757232 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e300ab9b0 con 0x7f3e300aecc0 2024-08-15T17:59:10.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e2e59c700 1 --2- 172.21.15.38:0/201757232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e300aecc0 0x7f3e3006df10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e300aeb30 tx=0x7f3e20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e538da2e5a4fff server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:10.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e2dd9b700 1 -- 172.21.15.38:0/201757232 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2000b640 con 0x7f3e300aecc0 2024-08-15T17:59:10.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e2dd9b700 1 -- 172.21.15.38:0/201757232 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2000b7a0 con 0x7f3e300aecc0 2024-08-15T17:59:10.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e2dd9b700 1 -- 172.21.15.38:0/201757232 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2000bb60 con 0x7f3e300aecc0 2024-08-15T17:59:10.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e34874700 1 -- 172.21.15.38:0/201757232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e300aecc0 msgr2=0x7f3e3006df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e34874700 1 --2- 172.21.15.38:0/201757232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e300aecc0 0x7f3e3006df10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e300aeb30 tx=0x7f3e20009250 comp rx=0 tx=0).stop 2024-08-15T17:59:10.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e34874700 1 -- 172.21.15.38:0/201757232 shutdown_connections 2024-08-15T17:59:10.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e34874700 1 --2- 172.21.15.38:0/201757232 >> v2:172.21.15.104:3300/0 conn(0x7f3e30159280 0x7f3e3006e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e34874700 1 --2- 172.21.15.38:0/201757232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e300aecc0 0x7f3e3006df10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e34874700 1 --2- 172.21.15.38:0/201757232 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e30158eb0 0x7f3e300aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e34874700 1 -- 172.21.15.38:0/201757232 >> 172.21.15.38:0/201757232 conn(0x7f3e300678d0 msgr2=0x7f3e30067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:10.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e34874700 1 -- 172.21.15.38:0/201757232 shutdown_connections 2024-08-15T17:59:10.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e34874700 1 -- 172.21.15.38:0/201757232 wait complete. 2024-08-15T17:59:10.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e34874700 1 Processor -- start 2024-08-15T17:59:10.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.533+0000 7f3e34874700 1 -- start start 2024-08-15T17:59:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e34874700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e300aecc0 0x7f3e3014cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e34874700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e30158eb0 0x7f3e30069400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e34874700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e30159280 0x7f3e30149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e34874700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3014d490 con 0x7f3e30159280 2024-08-15T17:59:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e34874700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e300cad80 con 0x7f3e30158eb0 2024-08-15T17:59:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e34874700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3016dd20 con 0x7f3e300aecc0 2024-08-15T17:59:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e30158eb0 0x7f3e30069400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2f59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e30159280 0x7f3e30149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2ed9d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e300aecc0 0x7f3e3014cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e30158eb0 0x7f3e30069400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:52684/0 (socket says 172.21.15.38:52684) 2024-08-15T17:59:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2e59c700 1 -- 172.21.15.38:0/1673592904 learned_addr learned my addr 172.21.15.38:0/1673592904 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2ed9d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e300aecc0 0x7f3e3014cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:59372/0 (socket says 172.21.15.38:59372) 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2f59e700 1 -- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e300aecc0 msgr2=0x7f3e3014cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2f59e700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e300aecc0 0x7f3e3014cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2f59e700 1 -- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e30158eb0 msgr2=0x7f3e30069400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2f59e700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e30158eb0 0x7f3e30069400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2f59e700 1 -- 172.21.15.38:0/1673592904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e200094f0 con 0x7f3e30159280 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2e59c700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e30158eb0 0x7f3e30069400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e2f59e700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e30159280 0x7f3e30149370 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009e50 tx=0x7f3e2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e16ffd700 1 -- 172.21.15.38:0/1673592904 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800caa0 con 0x7f3e30159280 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e16ffd700 1 -- 172.21.15.38:0/1673592904 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2800f040 con 0x7f3e30159280 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e34874700 1 -- 172.21.15.38:0/1673592904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3016cf60 con 0x7f3e30159280 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e16ffd700 1 -- 172.21.15.38:0/1673592904 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800ce50 con 0x7f3e30159280 2024-08-15T17:59:10.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.537+0000 7f3e34874700 1 -- 172.21.15.38:0/1673592904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3016d520 con 0x7f3e30159280 2024-08-15T17:59:10.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.541+0000 7f3e34874700 1 -- 172.21.15.38:0/1673592904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e300643a0 con 0x7f3e30159280 2024-08-15T17:59:10.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.541+0000 7f3e16ffd700 1 -- 172.21.15.38:0/1673592904 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3e28014690 con 0x7f3e30159280 2024-08-15T17:59:10.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.541+0000 7f3e16ffd700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3e1c060860 0x7f3e1c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.541+0000 7f3e16ffd700 1 -- 172.21.15.38:0/1673592904 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3e2807d760 con 0x7f3e30159280 2024-08-15T17:59:10.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.541+0000 7f3e2ed9d700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3e1c060860 0x7f3e1c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.541+0000 7f3e2ed9d700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3e1c060860 0x7f3e1c062d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009d20 tx=0x7f3e240091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:10.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.549+0000 7f3e16ffd700 1 -- 172.21.15.38:0/1673592904 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3e28051640 con 0x7f3e30159280 2024-08-15T17:59:10.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.749+0000 7f3e34874700 1 -- 172.21.15.38:0/1673592904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3e3006f5f0 con 0x7f3e30159280 2024-08-15T17:59:10.754 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-08-15T17:59:10.749+0000 7fa9f6684700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-15T17:59:10.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.757+0000 7f3e16ffd700 1 -- 172.21.15.38:0/1673592904 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3e280511d0 con 0x7f3e30159280 2024-08-15T17:59:10.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 -- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3e1c060860 msgr2=0x7f3e1c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3e1c060860 0x7f3e1c062d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009d20 tx=0x7f3e240091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 -- 172.21.15.38:0/1673592904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e30159280 msgr2=0x7f3e30149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e30159280 0x7f3e30149370 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009e50 tx=0x7f3e2800b040 comp rx=0 tx=0).stop 2024-08-15T17:59:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 -- 172.21.15.38:0/1673592904 shutdown_connections 2024-08-15T17:59:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3e1c060860 0x7f3e1c062d10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e30159280 0x7f3e30149370 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e30158eb0 0x7f3e30069400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 --2- 172.21.15.38:0/1673592904 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e300aecc0 0x7f3e3014cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 -- 172.21.15.38:0/1673592904 >> 172.21.15.38:0/1673592904 conn(0x7f3e300678d0 msgr2=0x7f3e3015cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 -- 172.21.15.38:0/1673592904 shutdown_connections 2024-08-15T17:59:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.761+0000 7f3e34874700 1 -- 172.21.15.38:0/1673592904 wait complete. 2024-08-15T17:59:10.767 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-15T17:59:10.769 INFO:tasks.ceph.mds.b.smithi104.stderr: -2207> 2024-08-15T17:58:40.580+0000 7f6f213e5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T17:59:10.779 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-08-15T17:59:10.795 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-08-15T17:59:10.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff00b2f4700 1 Processor -- start 2024-08-15T17:59:10.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff00b2f4700 1 -- start start 2024-08-15T17:59:10.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff00b2f4700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff00406e470 0x7ff0040aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff00b2f4700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff00406e840 0x7ff0040aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff00b2f4700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0040b1a40 con 0x7ff0040b0ef0 2024-08-15T17:59:10.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff00b2f4700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff004140f70 con 0x7ff00406e840 2024-08-15T17:59:10.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff00b2f4700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff004141250 con 0x7ff00406e470 2024-08-15T17:59:10.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff009af1700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff00406e840 0x7ff0040aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff009af1700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff00406e840 0x7ff0040aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35352/0 (socket says 172.21.15.38:35352) 2024-08-15T17:59:10.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff009af1700 1 -- 172.21.15.38:0/240638772 learned_addr learned my addr 172.21.15.38:0/240638772 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:10.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff00a2f2700 1 --2- 172.21.15.38:0/240638772 >> v2:172.21.15.104:3300/0 conn(0x7ff00406e470 0x7ff0040aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff009af1700 1 -- 172.21.15.38:0/240638772 >> v2:172.21.15.104:3300/0 conn(0x7ff00406e470 msgr2=0x7ff0040aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:10.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff009af1700 1 --2- 172.21.15.38:0/240638772 >> v2:172.21.15.104:3300/0 conn(0x7ff00406e470 0x7ff0040aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff009af1700 1 -- 172.21.15.38:0/240638772 >> v1:172.21.15.104:6789/0 conn(0x7ff0040b0ef0 legacy=0x7ff0040b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:10.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff009af1700 1 -- 172.21.15.38:0/240638772 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff004141530 con 0x7ff00406e840 2024-08-15T17:59:10.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff00a2f2700 1 --2- 172.21.15.38:0/240638772 >> v2:172.21.15.104:3300/0 conn(0x7ff00406e470 0x7ff0040aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:59:10.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff009af1700 1 --2- 172.21.15.38:0/240638772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00406e840 0x7ff0040aefc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff0000099b0 tx=0x7ff000009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420f4604e93fd528 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:10.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff0092f0700 1 -- 172.21.15.38:0/240638772 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000b7e0 con 0x7ff00406e840 2024-08-15T17:59:10.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff0092f0700 1 -- 172.21.15.38:0/240638772 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00000b940 con 0x7ff00406e840 2024-08-15T17:59:10.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff0092f0700 1 -- 172.21.15.38:0/240638772 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000bce0 con 0x7ff00406e840 2024-08-15T17:59:10.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff00b2f4700 1 -- 172.21.15.38:0/240638772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00406e840 msgr2=0x7ff0040aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.889+0000 7ff00b2f4700 1 --2- 172.21.15.38:0/240638772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00406e840 0x7ff0040aefc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff0000099b0 tx=0x7ff000009250 comp rx=0 tx=0).stop 2024-08-15T17:59:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 -- 172.21.15.38:0/240638772 shutdown_connections 2024-08-15T17:59:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 --2- 172.21.15.38:0/240638772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00406e840 0x7ff0040aefc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 --2- 172.21.15.38:0/240638772 >> v2:172.21.15.104:3300/0 conn(0x7ff00406e470 0x7ff0040aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 -- 172.21.15.38:0/240638772 >> 172.21.15.38:0/240638772 conn(0x7ff0040678a0 msgr2=0x7ff004067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 -- 172.21.15.38:0/240638772 shutdown_connections 2024-08-15T17:59:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 -- 172.21.15.38:0/240638772 wait complete. 2024-08-15T17:59:10.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 Processor -- start 2024-08-15T17:59:10.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 -- start start 2024-08-15T17:59:10.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00406e470 0x7ff00415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0040b0ef0 0x7ff0040c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0040c4f70 0x7ff004160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00b2f4700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040c2080 con 0x7ff00406e470 2024-08-15T17:59:10.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.893+0000 7ff00a2f2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00406e470 0x7ff00415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00a2f2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00406e470 0x7ff00415b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35356/0 (socket says 172.21.15.38:35356) 2024-08-15T17:59:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00a2f2700 1 -- 172.21.15.38:0/153868349 learned_addr learned my addr 172.21.15.38:0/153868349 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff004142ce0 con 0x7ff0040c4f70 2024-08-15T17:59:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040c55a0 con 0x7ff0040b0ef0 2024-08-15T17:59:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00aaf3700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0040c4f70 0x7ff004160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00aaf3700 1 -- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0040b0ef0 msgr2=0x7ff0040c4a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T17:59:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00aaf3700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0040b0ef0 0x7ff0040c4a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00aaf3700 1 -- 172.21.15.38:0/153868349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00406e470 msgr2=0x7ff00415b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00aaf3700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00406e470 0x7ff00415b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00aaf3700 1 -- 172.21.15.38:0/153868349 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0000094f0 con 0x7ff0040c4f70 2024-08-15T17:59:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00aaf3700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0040c4f70 0x7ff004160070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feffc00c810 tx=0x7feffc002620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7feffa7fc700 1 -- 172.21.15.38:0/153868349 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc012630 con 0x7ff0040c4f70 2024-08-15T17:59:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7feffa7fc700 1 -- 172.21.15.38:0/153868349 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feffc0127d0 con 0x7ff0040c4f70 2024-08-15T17:59:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0040c5880 con 0x7ff0040c4f70 2024-08-15T17:59:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0040c1d10 con 0x7ff0040c4f70 2024-08-15T17:59:10.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7feffa7fc700 1 -- 172.21.15.38:0/153868349 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc012b90 con 0x7ff0040c4f70 2024-08-15T17:59:10.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.897+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0040c2230 con 0x7ff0040c4f70 2024-08-15T17:59:10.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.901+0000 7feffa7fc700 1 -- 172.21.15.38:0/153868349 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7feffc024430 con 0x7ff0040c4f70 2024-08-15T17:59:10.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.901+0000 7feffa7fc700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefe8060890 0x7fefe8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:10.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.901+0000 7feffa7fc700 1 -- 172.21.15.38:0/153868349 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7feffc07ce20 con 0x7ff0040c4f70 2024-08-15T17:59:10.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.901+0000 7ff00a2f2700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefe8060890 0x7fefe8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:10.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.901+0000 7ff00a2f2700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefe8060890 0x7fefe8062d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feff4006fd0 tx=0x7feff4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:10.909+0000 7feffa7fc700 1 -- 172.21.15.38:0/153868349 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7feffc04d220 con 0x7ff0040c4f70 2024-08-15T17:59:11.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.109+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff0040c21f0 con 0x7ff0040c4f70 2024-08-15T17:59:11.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.761+0000 7feffa7fc700 1 -- 172.21.15.38:0/153868349 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7feffc050840 con 0x7ff0040c4f70 2024-08-15T17:59:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefe8060890 msgr2=0x7fefe8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefe8060890 0x7fefe8062d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feff4006fd0 tx=0x7feff4008040 comp rx=0 tx=0).stop 2024-08-15T17:59:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0040c4f70 msgr2=0x7ff004160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0040c4f70 0x7ff004160070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feffc00c810 tx=0x7feffc002620 comp rx=0 tx=0).stop 2024-08-15T17:59:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 shutdown_connections 2024-08-15T17:59:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fefe8060890 0x7fefe8062d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0040c4f70 0x7ff004160070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:11.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0040b0ef0 0x7ff0040c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:11.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 --2- 172.21.15.38:0/153868349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00406e470 0x7ff00415b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:11.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 >> 172.21.15.38:0/153868349 conn(0x7ff0040678a0 msgr2=0x7ff00415eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:11.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 shutdown_connections 2024-08-15T17:59:11.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.769+0000 7ff00b2f4700 1 -- 172.21.15.38:0/153868349 wait complete. 2024-08-15T17:59:11.788 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-15T17:59:11.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.909+0000 7ff4af59e700 1 Processor -- start 2024-08-15T17:59:11.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.909+0000 7ff4af59e700 1 -- start start 2024-08-15T17:59:11.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.909+0000 7ff4af59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff4b00b26d0 0x7ff4b00b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:11.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.909+0000 7ff4af59e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff4b00cded0 0x7ff4b00ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:11.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.909+0000 7ff4af59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b00c5b40 con 0x7ff4b00c5fd0 2024-08-15T17:59:11.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.909+0000 7ff4af59e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b00b2aa0 con 0x7ff4b00cded0 2024-08-15T17:59:11.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.909+0000 7ff4af59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b0159820 con 0x7ff4b00b26d0 2024-08-15T17:59:11.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.909+0000 7ff4aed9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff4b00cded0 0x7ff4b00ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:11.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.909+0000 7ff4aed9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff4b00cded0 0x7ff4b00ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35372/0 (socket says 172.21.15.38:35372) 2024-08-15T17:59:11.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.909+0000 7ff4aed9d700 1 -- 172.21.15.38:0/198920934 learned_addr learned my addr 172.21.15.38:0/198920934 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:11.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.909+0000 7ff4ae59c700 1 --2- 172.21.15.38:0/198920934 >> v2:172.21.15.104:3300/0 conn(0x7ff4b00b26d0 0x7ff4b00b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:11.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4aed9d700 1 -- 172.21.15.38:0/198920934 >> v2:172.21.15.104:3300/0 conn(0x7ff4b00b26d0 msgr2=0x7ff4b00b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:11.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4aed9d700 1 --2- 172.21.15.38:0/198920934 >> v2:172.21.15.104:3300/0 conn(0x7ff4b00b26d0 0x7ff4b00b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:11.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4aed9d700 1 -- 172.21.15.38:0/198920934 >> v1:172.21.15.104:6789/0 conn(0x7ff4b00c5fd0 legacy=0x7ff4b00c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:11.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4aed9d700 1 -- 172.21.15.38:0/198920934 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b00c4f30 con 0x7ff4b00cded0 2024-08-15T17:59:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4aed9d700 1 --2- 172.21.15.38:0/198920934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b00cded0 0x7ff4b00ce330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009f20 tx=0x7ff4a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68e7eaf30083c46 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4ad59a700 1 -- 172.21.15.38:0/198920934 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800e070 con 0x7ff4b00cded0 2024-08-15T17:59:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4ad59a700 1 -- 172.21.15.38:0/198920934 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a800ba40 con 0x7ff4b00cded0 2024-08-15T17:59:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4ad59a700 1 -- 172.21.15.38:0/198920934 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80092e0 con 0x7ff4b00cded0 2024-08-15T17:59:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4af59e700 1 -- 172.21.15.38:0/198920934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b00cded0 msgr2=0x7ff4b00ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4af59e700 1 --2- 172.21.15.38:0/198920934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b00cded0 0x7ff4b00ce330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009f20 tx=0x7ff4a8009840 comp rx=0 tx=0).stop 2024-08-15T17:59:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4af59e700 1 -- 172.21.15.38:0/198920934 shutdown_connections 2024-08-15T17:59:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4af59e700 1 --2- 172.21.15.38:0/198920934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b00cded0 0x7ff4b00ce330 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4af59e700 1 --2- 172.21.15.38:0/198920934 >> v2:172.21.15.104:3300/0 conn(0x7ff4b00b26d0 0x7ff4b00b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4af59e700 1 -- 172.21.15.38:0/198920934 >> 172.21.15.38:0/198920934 conn(0x7ff4b0067590 msgr2=0x7ff4b00d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4af59e700 1 -- 172.21.15.38:0/198920934 shutdown_connections 2024-08-15T17:59:11.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.913+0000 7ff4af59e700 1 -- 172.21.15.38:0/198920934 wait complete. 2024-08-15T17:59:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4af59e700 1 Processor -- start 2024-08-15T17:59:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4af59e700 1 -- start start 2024-08-15T17:59:11.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4af59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4b00b26d0 0x7ff4b01549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:11.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4af59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4b00c5fd0 0x7ff4b0154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:11.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4af59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b00cded0 0x7ff4b009d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:11.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4af59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b009c9f0 con 0x7ff4b00cded0 2024-08-15T17:59:11.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4af59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c92b0 con 0x7ff4b00b26d0 2024-08-15T17:59:11.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4af59e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c8fd0 con 0x7ff4b00c5fd0 2024-08-15T17:59:11.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4aed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b00cded0 0x7ff4b009d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:11.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4aed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b00cded0 0x7ff4b009d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35378/0 (socket says 172.21.15.38:35378) 2024-08-15T17:59:11.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4aed9d700 1 -- 172.21.15.38:0/25527403 learned_addr learned my addr 172.21.15.38:0/25527403 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:11.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4ae59c700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4b00b26d0 0x7ff4b01549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:11.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4add9b700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4b00c5fd0 0x7ff4b0154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4aed9d700 1 -- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4b00c5fd0 msgr2=0x7ff4b0154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4aed9d700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4b00c5fd0 0x7ff4b0154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4aed9d700 1 -- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4b00b26d0 msgr2=0x7ff4b01549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4aed9d700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4b00b26d0 0x7ff4b01549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4aed9d700 1 -- 172.21.15.38:0/25527403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a8009ae0 con 0x7ff4b00cded0 2024-08-15T17:59:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4aed9d700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b00cded0 0x7ff4b009d700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00893f0 tx=0x7ff4a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4a67fc700 1 -- 172.21.15.38:0/25527403 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8008ce0 con 0x7ff4b00cded0 2024-08-15T17:59:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4a67fc700 1 -- 172.21.15.38:0/25527403 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a8008e80 con 0x7ff4b00cded0 2024-08-15T17:59:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4af59e700 1 -- 172.21.15.38:0/25527403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b009cc20 con 0x7ff4b00cded0 2024-08-15T17:59:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4a67fc700 1 -- 172.21.15.38:0/25527403 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80266b0 con 0x7ff4b00cded0 2024-08-15T17:59:11.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.917+0000 7ff4af59e700 1 -- 172.21.15.38:0/25527403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b009d1e0 con 0x7ff4b00cded0 2024-08-15T17:59:11.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.921+0000 7ff4af59e700 1 -- 172.21.15.38:0/25527403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b009d3d0 con 0x7ff4b00cded0 2024-08-15T17:59:11.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.921+0000 7ff4a67fc700 1 -- 172.21.15.38:0/25527403 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff4a8003040 con 0x7ff4b00cded0 2024-08-15T17:59:11.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.921+0000 7ff4a67fc700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff490060860 0x7ff490062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:11.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.921+0000 7ff4a67fc700 1 -- 172.21.15.38:0/25527403 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7ff4a807e5b0 con 0x7ff4b00cded0 2024-08-15T17:59:11.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.921+0000 7ff4ae59c700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff490060860 0x7ff490062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:11.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.925+0000 7ff4ae59c700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff490060860 0x7ff490062d10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff498006fd0 tx=0x7ff498008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:11.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:11.933+0000 7ff4a67fc700 1 -- 172.21.15.38:0/25527403 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff4a804ea60 con 0x7ff4b00cded0 2024-08-15T17:59:12.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.101+0000 7ff4af59e700 1 -- 172.21.15.38:0/25527403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff4b01547b0 con 0x7ff4b00cded0 2024-08-15T17:59:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.781+0000 7ff4a67fc700 1 -- 172.21.15.38:0/25527403 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff4a8052080 con 0x7ff4b00cded0 2024-08-15T17:59:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 -- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff490060860 msgr2=0x7ff490062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff490060860 0x7ff490062d10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff498006fd0 tx=0x7ff498008040 comp rx=0 tx=0).stop 2024-08-15T17:59:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 -- 172.21.15.38:0/25527403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b00cded0 msgr2=0x7ff4b009d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b00cded0 0x7ff4b009d700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00893f0 tx=0x7ff4a8018000 comp rx=0 tx=0).stop 2024-08-15T17:59:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 -- 172.21.15.38:0/25527403 shutdown_connections 2024-08-15T17:59:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff490060860 0x7ff490062d10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b00cded0 0x7ff4b009d700 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4b00c5fd0 0x7ff4b0154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 --2- 172.21.15.38:0/25527403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4b00b26d0 0x7ff4b01549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 -- 172.21.15.38:0/25527403 >> 172.21.15.38:0/25527403 conn(0x7ff4b0067590 msgr2=0x7ff4b0068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 -- 172.21.15.38:0/25527403 shutdown_connections 2024-08-15T17:59:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.785+0000 7ff4af59e700 1 -- 172.21.15.38:0/25527403 wait complete. 2024-08-15T17:59:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-08-15T17:59:12.803 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-15T17:59:12.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0702927700 1 Processor -- start 2024-08-15T17:59:12.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0702927700 1 -- start start 2024-08-15T17:59:12.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0702927700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc0ceee0 0x7f06fc163f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:12.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0702927700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc0c5b40 con 0x7f06fc0b2e90 2024-08-15T17:59:12.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0702927700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc0b2aa0 con 0x7f06fc0b26d0 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0702927700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0b3310 con 0x7f06fc0ceee0 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0701124700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f06fc0b2e90 0x7f06fc0c66e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35352/0 (socket says 172.21.15.38:35352) 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0701124700 1 -- 172.21.15.38:0/2865055709 learned_addr learned my addr 172.21.15.38:0/2865055709 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0702126700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc0ceee0 0x7f06fc163f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871891461 0 0) 0x7f06fc0c5b40 con 0x7f06fc0b2e90 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06e8003570 con 0x7f06fc0b2e90 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163785525 0 0) 0x7f06fc0b2aa0 con 0x7f06fc0b26d0 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06fc0c5b40 con 0x7f06fc0b26d0 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967636098 0 0) 0x7f06e8003570 con 0x7f06fc0b2e90 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc0ceee0 msgr2=0x7f06fc163f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc0ceee0 0x7f06fc163f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 >> v1:172.21.15.104:6789/0 conn(0x7f06fc0b26d0 legacy=0x7f06fc0c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:12.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc159820 con 0x7f06fc0b2e90 2024-08-15T17:59:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f06f00015f0 con 0x7f06fc0b2e90 2024-08-15T17:59:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 >> v1:172.21.15.38:6789/0 conn(0x7f06fc0b2e90 legacy=0x7f06fc0c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc0ceee0 0x7f06e8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e80041d0 0x7f06e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06e8004d50 0x7f06e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0b3310 con 0x7f06e80041d0 2024-08-15T17:59:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e800bbf0 con 0x7f06e8004d50 2024-08-15T17:59:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e800bdb0 con 0x7f06fc0ceee0 2024-08-15T17:59:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0701124700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e80041d0 0x7f06e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0701925700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc0ceee0 0x7f06e8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.921+0000 7f0702126700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06e8004d50 0x7f06e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0701124700 1 -- 172.21.15.38:0/2865055709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc0ceee0 msgr2=0x7f06e8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0701124700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc0ceee0 0x7f06e8003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0701124700 1 -- 172.21.15.38:0/2865055709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06e8004d50 msgr2=0x7f06e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0701124700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06e8004d50 0x7f06e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0701124700 1 -- 172.21.15.38:0/2865055709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e8003570 con 0x7f06e80041d0 2024-08-15T17:59:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0701124700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e80041d0 0x7f06e8004810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c210 tx=0x7f06f0007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a866d2974b8275b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f000cae0 con 0x7f06e80041d0 2024-08-15T17:59:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f000cc40 con 0x7f06e80041d0 2024-08-15T17:59:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0700923700 1 -- 172.21.15.38:0/2865055709 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f00070d0 con 0x7f06e80041d0 2024-08-15T17:59:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0702927700 1 -- 172.21.15.38:0/2865055709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e80041d0 msgr2=0x7f06e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0702927700 1 --2- 172.21.15.38:0/2865055709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e80041d0 0x7f06e8004810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c210 tx=0x7f06f0007810 comp rx=0 tx=0).stop 2024-08-15T17:59:12.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0702126700 1 -- 172.21.15.38:0/2865055709 reap_dead start 2024-08-15T17:59:12.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0702927700 1 -- 172.21.15.38:0/2865055709 shutdown_connections 2024-08-15T17:59:12.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0702927700 1 -- 172.21.15.38:0/2865055709 >> 172.21.15.38:0/2865055709 conn(0x7f06fc067590 msgr2=0x7f06fc0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:12.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0702927700 1 -- 172.21.15.38:0/2865055709 shutdown_connections 2024-08-15T17:59:12.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.925+0000 7f0702927700 1 -- 172.21.15.38:0/2865055709 wait complete. 2024-08-15T17:59:12.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0702927700 1 Processor -- start 2024-08-15T17:59:12.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0702927700 1 -- start start 2024-08-15T17:59:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0702927700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06fc152580 0x7f06fc1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0702927700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc15ea10 0x7f06fc152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0702927700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06fc152e90 0x7f06fc15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0702927700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc159f10 con 0x7f06fc152580 2024-08-15T17:59:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0702927700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0c7240 con 0x7f06fc152e90 2024-08-15T17:59:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0702927700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0c74b0 con 0x7f06fc15ea10 2024-08-15T17:59:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0701925700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06fc152580 0x7f06fc1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0701925700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06fc152580 0x7f06fc1599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35398/0 (socket says 172.21.15.38:35398) 2024-08-15T17:59:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0701925700 1 -- 172.21.15.38:0/130613783 learned_addr learned my addr 172.21.15.38:0/130613783 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0702126700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06fc152e90 0x7f06fc15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0701124700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc15ea10 0x7f06fc152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0701925700 1 -- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc15ea10 msgr2=0x7f06fc152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0701925700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc15ea10 0x7f06fc152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0701925700 1 -- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06fc152e90 msgr2=0x7f06fc15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0701925700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06fc152e90 0x7f06fc15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:12.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.929+0000 7f0701925700 1 -- 172.21.15.38:0/130613783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc0c7790 con 0x7f06fc152580 2024-08-15T17:59:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.933+0000 7f0701925700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06fc152580 0x7f06fc1599d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f06f8002c00 tx=0x7f06f8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.933+0000 7f06edffb700 1 -- 172.21.15.38:0/130613783 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800f920 con 0x7f06fc152580 2024-08-15T17:59:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.933+0000 7f06edffb700 1 -- 172.21.15.38:0/130613783 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f800fa80 con 0x7f06fc152580 2024-08-15T17:59:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.933+0000 7f06edffb700 1 -- 172.21.15.38:0/130613783 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800fe40 con 0x7f06fc152580 2024-08-15T17:59:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.933+0000 7f0702927700 1 -- 172.21.15.38:0/130613783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc0c7a70 con 0x7f06fc152580 2024-08-15T17:59:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.933+0000 7f0702927700 1 -- 172.21.15.38:0/130613783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc15ee80 con 0x7f06fc152580 2024-08-15T17:59:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.933+0000 7f06edffb700 1 -- 172.21.15.38:0/130613783 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f06f8013600 con 0x7f06fc152580 2024-08-15T17:59:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.933+0000 7f0702927700 1 -- 172.21.15.38:0/130613783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc0c7c60 con 0x7f06fc152580 2024-08-15T17:59:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.933+0000 7f06edffb700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06e0060860 0x7f06e0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.933+0000 7f06edffb700 1 -- 172.21.15.38:0/130613783 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f06f807d0f0 con 0x7f06fc152580 2024-08-15T17:59:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.933+0000 7f0701124700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06e0060860 0x7f06e0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:12.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.937+0000 7f0701124700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06e0060860 0x7f06e0062d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06fc061060 tx=0x7f06f0015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:12.941+0000 7f06edffb700 1 -- 172.21.15.38:0/130613783 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f06f804d680 con 0x7f06fc152580 2024-08-15T17:59:13.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.105+0000 7f0702927700 1 -- 172.21.15.38:0/130613783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06fc0644b0 con 0x7f06fc152580 2024-08-15T17:59:13.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.793+0000 7f06edffb700 1 -- 172.21.15.38:0/130613783 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f06f8050ca0 con 0x7f06fc152580 2024-08-15T17:59:13.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 -- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06e0060860 msgr2=0x7f06e0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:13.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06e0060860 0x7f06e0062d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06fc061060 tx=0x7f06f0015000 comp rx=0 tx=0).stop 2024-08-15T17:59:13.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 -- 172.21.15.38:0/130613783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06fc152580 msgr2=0x7f06fc1599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:13.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06fc152580 0x7f06fc1599d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f06f8002c00 tx=0x7f06f8007d30 comp rx=0 tx=0).stop 2024-08-15T17:59:13.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 -- 172.21.15.38:0/130613783 shutdown_connections 2024-08-15T17:59:13.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06e0060860 0x7f06e0062d10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:13.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06fc152e90 0x7f06fc15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:13.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06fc15ea10 0x7f06fc152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:13.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 --2- 172.21.15.38:0/130613783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06fc152580 0x7f06fc1599d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:13.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 -- 172.21.15.38:0/130613783 >> 172.21.15.38:0/130613783 conn(0x7f06fc067590 msgr2=0x7f06fc0cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:13.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 -- 172.21.15.38:0/130613783 shutdown_connections 2024-08-15T17:59:13.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.801+0000 7f0702927700 1 -- 172.21.15.38:0/130613783 wait complete. 2024-08-15T17:59:13.805 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-08-15T17:59:13.818 INFO:tasks.ceph.mds.a:Restarting daemon 2024-08-15T17:59:13.818 INFO:tasks.ceph.mds.a:Stopping old one... 2024-08-15T17:59:13.819 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-08-15T17:59:13.819 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:59:13.819 INFO:tasks.ceph.mds.c:Restarting daemon 2024-08-15T17:59:13.819 INFO:tasks.ceph.mds.c:Stopping old one... 2024-08-15T17:59:13.819 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-08-15T17:59:13.819 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:59:13.820 INFO:tasks.ceph.mds.b:Restarting daemon 2024-08-15T17:59:13.820 INFO:tasks.ceph.mds.b:Stopping old one... 2024-08-15T17:59:13.820 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-08-15T17:59:13.820 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:59:13.820 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-15T17:59:13.820 INFO:tasks.ceph.mds.d:Stopping old one... 2024-08-15T17:59:13.821 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-15T17:59:13.821 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:59:13.830 INFO:tasks.ceph.mds.a:Stopped 2024-08-15T17:59:13.830 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-08-15T17:59:13.831 INFO:tasks.ceph.mds.c:Stopped 2024-08-15T17:59:13.831 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-08-15T17:59:13.833 INFO:tasks.ceph.mds.a:Started 2024-08-15T17:59:13.833 DEBUG:teuthology.parallel:result is None 2024-08-15T17:59:13.834 INFO:tasks.ceph.mds.b:Stopped 2024-08-15T17:59:13.835 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-08-15T17:59:13.836 INFO:tasks.ceph.mds.d:Stopped 2024-08-15T17:59:13.836 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-15T17:59:13.837 INFO:tasks.ceph.mds.c:Started 2024-08-15T17:59:13.837 INFO:tasks.ceph.mds.b:Started 2024-08-15T17:59:13.837 DEBUG:teuthology.parallel:result is None 2024-08-15T17:59:13.838 DEBUG:teuthology.parallel:result is None 2024-08-15T17:59:13.838 INFO:tasks.ceph.mds.d:Started 2024-08-15T17:59:13.838 DEBUG:teuthology.parallel:result is None 2024-08-15T17:59:13.839 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-08-15T17:59:13.900 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-08-15T17:59:13.903 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-08-15T17:59:13.905 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-08-15T17:59:13.907 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-08-15T17:59:13.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.945+0000 7f79be989700 1 Processor -- start 2024-08-15T17:59:13.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.945+0000 7f79be989700 1 -- start start 2024-08-15T17:59:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.945+0000 7f79be989700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f79b8159040 0x7f79b80cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.945+0000 7f79be989700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b815d5b0 0x7f79b80d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.945+0000 7f79be989700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b80cf190 con 0x7f79b80d0010 2024-08-15T17:59:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.945+0000 7f79be989700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b80d09a0 con 0x7f79b8159040 2024-08-15T17:59:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.945+0000 7f79be989700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80ce5e0 con 0x7f79b815d5b0 2024-08-15T17:59:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.945+0000 7f79bd987700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f79b8159040 0x7f79b80cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.945+0000 7f79bd186700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f79b80d0010 0x7f79b815cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35376/0 (socket says 172.21.15.38:35376) 2024-08-15T17:59:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79bd186700 1 -- 172.21.15.38:0/3592849088 learned_addr learned my addr 172.21.15.38:0/3592849088 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79be188700 1 --2- 172.21.15.38:0/3592849088 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b815d5b0 0x7f79b80d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79bd987700 1 -- 172.21.15.38:0/3592849088 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b815d5b0 msgr2=0x7f79b80d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79bd987700 1 --2- 172.21.15.38:0/3592849088 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b815d5b0 0x7f79b80d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:13.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79bd987700 1 -- 172.21.15.38:0/3592849088 >> v1:172.21.15.38:6789/0 conn(0x7f79b80d0010 legacy=0x7f79b815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:13.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79bd987700 1 -- 172.21.15.38:0/3592849088 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b80ce860 con 0x7f79b8159040 2024-08-15T17:59:13.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79bd987700 1 --2- 172.21.15.38:0/3592849088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b8159040 0x7f79b80cf620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009e60 tx=0x7f79b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8ee93622a70eac server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:13.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79bc985700 1 -- 172.21.15.38:0/3592849088 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b000e070 con 0x7f79b8159040 2024-08-15T17:59:13.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79bc985700 1 -- 172.21.15.38:0/3592849088 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b000bc30 con 0x7f79b8159040 2024-08-15T17:59:13.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79bc985700 1 -- 172.21.15.38:0/3592849088 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0012430 con 0x7f79b8159040 2024-08-15T17:59:13.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79be989700 1 -- 172.21.15.38:0/3592849088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b8159040 msgr2=0x7f79b80cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:13.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79be989700 1 --2- 172.21.15.38:0/3592849088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b8159040 0x7f79b80cf620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009e60 tx=0x7f79b0009780 comp rx=0 tx=0).stop 2024-08-15T17:59:13.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79be989700 1 -- 172.21.15.38:0/3592849088 shutdown_connections 2024-08-15T17:59:13.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79be989700 1 --2- 172.21.15.38:0/3592849088 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b815d5b0 0x7f79b80d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:13.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79be989700 1 --2- 172.21.15.38:0/3592849088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b8159040 0x7f79b80cf620 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:13.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79be989700 1 -- 172.21.15.38:0/3592849088 >> 172.21.15.38:0/3592849088 conn(0x7f79b8068250 msgr2=0x7f79b8068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:13.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79be989700 1 -- 172.21.15.38:0/3592849088 shutdown_connections 2024-08-15T17:59:13.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79be989700 1 -- 172.21.15.38:0/3592849088 wait complete. 2024-08-15T17:59:13.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79be989700 1 Processor -- start 2024-08-15T17:59:13.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.949+0000 7f79be989700 1 -- start start 2024-08-15T17:59:13.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79be989700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b80d0010 0x7f79b8169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79be989700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79b8159040 0x7f79b816c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79be989700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b815d5b0 0x7f79b80b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79be989700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80b1df0 con 0x7f79b80d0010 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79be989700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80b0930 con 0x7f79b8159040 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79be989700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b816cb00 con 0x7f79b815d5b0 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79bd987700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b80d0010 0x7f79b8169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79bd987700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b80d0010 0x7f79b8169870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35452/0 (socket says 172.21.15.38:35452) 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79bd987700 1 -- 172.21.15.38:0/912063686 learned_addr learned my addr 172.21.15.38:0/912063686 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79bd186700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79b8159040 0x7f79b816c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79be188700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b815d5b0 0x7f79b80b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79bd987700 1 -- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b815d5b0 msgr2=0x7f79b80b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:13.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79bd987700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b815d5b0 0x7f79b80b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:13.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79bd987700 1 -- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79b8159040 msgr2=0x7f79b816c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:13.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79bd987700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79b8159040 0x7f79b816c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:13.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79bd987700 1 -- 172.21.15.38:0/912063686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b0009a20 con 0x7f79b80d0010 2024-08-15T17:59:13.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79bd186700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79b8159040 0x7f79b816c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:59:13.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79bd987700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b80d0010 0x7f79b8169870 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79b00097c0 tx=0x7f79b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:13.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79adffb700 1 -- 172.21.15.38:0/912063686 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0012e80 con 0x7f79b80d0010 2024-08-15T17:59:13.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79adffb700 1 -- 172.21.15.38:0/912063686 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b0027470 con 0x7f79b80d0010 2024-08-15T17:59:13.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79be989700 1 -- 172.21.15.38:0/912063686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b80afa40 con 0x7f79b80d0010 2024-08-15T17:59:13.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79adffb700 1 -- 172.21.15.38:0/912063686 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0027830 con 0x7f79b80d0010 2024-08-15T17:59:13.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.953+0000 7f79be989700 1 -- 172.21.15.38:0/912063686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b80b0000 con 0x7f79b80d0010 2024-08-15T17:59:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.957+0000 7f79be989700 1 -- 172.21.15.38:0/912063686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b80643a0 con 0x7f79b80d0010 2024-08-15T17:59:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.957+0000 7f79adffb700 1 -- 172.21.15.38:0/912063686 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f79b0024070 con 0x7f79b80d0010 2024-08-15T17:59:13.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.957+0000 7f79adffb700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f799c060890 0x7f799c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:13.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.957+0000 7f79adffb700 1 -- 172.21.15.38:0/912063686 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f79b007f7d0 con 0x7f79b80d0010 2024-08-15T17:59:13.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.957+0000 7f79bd186700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f799c060890 0x7f799c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:13.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.961+0000 7f79bd186700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f799c060890 0x7f799c062d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f79b81549b0 tx=0x7f79a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:13.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:13.965+0000 7f79adffb700 1 -- 172.21.15.38:0/912063686 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f79b004ff40 con 0x7f79b80d0010 2024-08-15T17:59:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.137+0000 7f79be989700 1 -- 172.21.15.38:0/912063686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f79b80b3480 con 0x7f79b80d0010 2024-08-15T17:59:14.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.821+0000 7f79adffb700 1 -- 172.21.15.38:0/912063686 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7f79b0053560 con 0x7f79b80d0010 2024-08-15T17:59:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.825+0000 7f79be989700 1 -- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f799c060890 msgr2=0x7f799c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.825+0000 7f79be989700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f799c060890 0x7f799c062d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f79b81549b0 tx=0x7f79a8009040 comp rx=0 tx=0).stop 2024-08-15T17:59:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.825+0000 7f79be989700 1 -- 172.21.15.38:0/912063686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b80d0010 msgr2=0x7f79b8169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.825+0000 7f79be989700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b80d0010 0x7f79b8169870 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79b00097c0 tx=0x7f79b0019000 comp rx=0 tx=0).stop 2024-08-15T17:59:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.825+0000 7f79be989700 1 -- 172.21.15.38:0/912063686 shutdown_connections 2024-08-15T17:59:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.825+0000 7f79be989700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f799c060890 0x7f799c062d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.825+0000 7f79be989700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b815d5b0 0x7f79b80b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.825+0000 7f79be989700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79b8159040 0x7f79b816c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.825+0000 7f79be989700 1 --2- 172.21.15.38:0/912063686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b80d0010 0x7f79b8169870 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.825+0000 7f79be989700 1 -- 172.21.15.38:0/912063686 >> 172.21.15.38:0/912063686 conn(0x7f79b8068250 msgr2=0x7f79b8153740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.829+0000 7f79be989700 1 -- 172.21.15.38:0/912063686 shutdown_connections 2024-08-15T17:59:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.829+0000 7f79be989700 1 -- 172.21.15.38:0/912063686 wait complete. 2024-08-15T17:59:14.832 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-08-15T17:59:14.847 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-08-15T17:59:14.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.957+0000 7f605f59e700 1 Processor -- start 2024-08-15T17:59:14.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.957+0000 7f605f59e700 1 -- start start 2024-08-15T17:59:14.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f60601553e0 0x7f60600af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:14.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f60600686a0 0x7f6060068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:14.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6060155c00 con 0x7f60601557b0 2024-08-15T17:59:14.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6060069020 con 0x7f60600686a0 2024-08-15T17:59:14.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6060155e30 con 0x7f60601553e0 2024-08-15T17:59:14.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f6057fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f60601557b0 0x7f60600b1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35388/0 (socket says 172.21.15.38:35388) 2024-08-15T17:59:14.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605ed9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f60600686a0 0x7f6060068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:14.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f6057fff700 1 -- 172.21.15.38:0/3761621709 learned_addr learned my addr 172.21.15.38:0/3761621709 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:14.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605e59c700 1 --2- 172.21.15.38:0/3761621709 >> v2:172.21.15.104:3300/0 conn(0x7f60601553e0 0x7f60600af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:14.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605dd9b700 1 -- 172.21.15.38:0/3761621709 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3519905140 0 0) 0x7f6060155c00 con 0x7f60601557b0 2024-08-15T17:59:14.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605dd9b700 1 -- 172.21.15.38:0/3761621709 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f604c003570 con 0x7f60601557b0 2024-08-15T17:59:14.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605dd9b700 1 -- 172.21.15.38:0/3761621709 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3326936862 0 0) 0x7f604c003570 con 0x7f60601557b0 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605ed9d700 1 -- 172.21.15.38:0/3761621709 >> v2:172.21.15.104:3300/0 conn(0x7f60601553e0 msgr2=0x7f60600af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605ed9d700 1 --2- 172.21.15.38:0/3761621709 >> v2:172.21.15.104:3300/0 conn(0x7f60601553e0 0x7f60600af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605ed9d700 1 -- 172.21.15.38:0/3761621709 >> v1:172.21.15.38:6789/0 conn(0x7f60601557b0 legacy=0x7f60600b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605ed9d700 1 -- 172.21.15.38:0/3761621709 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6060156110 con 0x7f60600686a0 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605ed9d700 1 --2- 172.21.15.38:0/3761621709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60600686a0 0x7f6060068ae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6060070ce0 tx=0x7f6058009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0549404e06a052 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605dd9b700 1 -- 172.21.15.38:0/3761621709 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605800e070 con 0x7f60600686a0 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605dd9b700 1 -- 172.21.15.38:0/3761621709 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605800b850 con 0x7f60600686a0 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605dd9b700 1 -- 172.21.15.38:0/3761621709 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048001470 con 0x7f60600686a0 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 -- 172.21.15.38:0/3761621709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60600686a0 msgr2=0x7f6060068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 --2- 172.21.15.38:0/3761621709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60600686a0 0x7f6060068ae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6060070ce0 tx=0x7f6058009780 comp rx=0 tx=0).stop 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 -- 172.21.15.38:0/3761621709 shutdown_connections 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 --2- 172.21.15.38:0/3761621709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60600686a0 0x7f6060068ae0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:14.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 --2- 172.21.15.38:0/3761621709 >> v2:172.21.15.104:3300/0 conn(0x7f60601553e0 0x7f60600af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:14.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 -- 172.21.15.38:0/3761621709 >> 172.21.15.38:0/3761621709 conn(0x7f606006d800 msgr2=0x7f606006dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:14.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 -- 172.21.15.38:0/3761621709 shutdown_connections 2024-08-15T17:59:14.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.961+0000 7f605f59e700 1 -- 172.21.15.38:0/3761621709 wait complete. 2024-08-15T17:59:14.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605f59e700 1 Processor -- start 2024-08-15T17:59:14.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605f59e700 1 -- start start 2024-08-15T17:59:14.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605f59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60600686a0 0x7f6060158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:14.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605f59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60601553e0 0x7f6060152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:14.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605f59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f60601557b0 0x7f6060152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:14.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605f59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606013e9a0 con 0x7f60600686a0 2024-08-15T17:59:14.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605f59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606013dd90 con 0x7f60601553e0 2024-08-15T17:59:14.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605f59e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606013e070 con 0x7f60601557b0 2024-08-15T17:59:14.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60600686a0 0x7f6060158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:14.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f6057fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60601553e0 0x7f6060152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:14.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60600686a0 0x7f6060158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35470/0 (socket says 172.21.15.38:35470) 2024-08-15T17:59:14.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605e59c700 1 -- 172.21.15.38:0/2824137201 learned_addr learned my addr 172.21.15.38:0/2824137201 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:14.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605ed9d700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f60601557b0 0x7f6060152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:14.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605e59c700 1 -- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f60601557b0 msgr2=0x7f6060152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605e59c700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f60601557b0 0x7f6060152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605e59c700 1 -- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60601553e0 msgr2=0x7f6060152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605e59c700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60601553e0 0x7f6060152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605e59c700 1 -- 172.21.15.38:0/2824137201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6058009a20 con 0x7f60600686a0 2024-08-15T17:59:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605e59c700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60600686a0 0x7f6060158390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f605000d890 tx=0x7f6050008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f60567fc700 1 -- 172.21.15.38:0/2824137201 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60500125f0 con 0x7f60600686a0 2024-08-15T17:59:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f60567fc700 1 -- 172.21.15.38:0/2824137201 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6050012750 con 0x7f60600686a0 2024-08-15T17:59:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605f59e700 1 -- 172.21.15.38:0/2824137201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606013e350 con 0x7f60600686a0 2024-08-15T17:59:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f60567fc700 1 -- 172.21.15.38:0/2824137201 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050012af0 con 0x7f60600686a0 2024-08-15T17:59:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.965+0000 7f605f59e700 1 -- 172.21.15.38:0/2824137201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6060154c00 con 0x7f60600686a0 2024-08-15T17:59:14.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.969+0000 7f605f59e700 1 -- 172.21.15.38:0/2824137201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60600638f0 con 0x7f60600686a0 2024-08-15T17:59:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.969+0000 7f60567fc700 1 -- 172.21.15.38:0/2824137201 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f605001e460 con 0x7f60600686a0 2024-08-15T17:59:14.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.969+0000 7f60567fc700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6040060890 0x7f6040062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:14.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.969+0000 7f60567fc700 1 -- 172.21.15.38:0/2824137201 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f605007d940 con 0x7f60600686a0 2024-08-15T17:59:14.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.969+0000 7f6057fff700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6040060890 0x7f6040062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:14.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.973+0000 7f6057fff700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6040060890 0x7f6040062d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f60600acad0 tx=0x7f6048001a00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:14.973+0000 7f60567fc700 1 -- 172.21.15.38:0/2824137201 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f605004e0c0 con 0x7f60600686a0 2024-08-15T17:59:15.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.189+0000 7f605f59e700 1 -- 172.21.15.38:0/2824137201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f606013e7d0 con 0x7f60600686a0 2024-08-15T17:59:15.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.189+0000 7f60567fc700 1 -- 172.21.15.38:0/2824137201 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f60500516e0 con 0x7f60600686a0 2024-08-15T17:59:15.194 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:15.194 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T17:59:15.194 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-08-15T17:59:15.194 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.194 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-08-15T17:59:15.194 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCQQb5mQfBdFBAA3Gx7jZsEWeDgeuSMICyUmg==", 2024-08-15T17:59:15.194 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.194 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-08-15T17:59:15.194 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-08-15T17:59:15.194 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-08-15T17:59:15.194 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCQQb5msU9/GxAAtm3HTiBfvo2KZQ7FOe+yIw==", 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.195 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCQQb5m9LPaFxAAY+1BXg+8eqRcz8RPF4rUbg==", 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCQQb5msSGnHhAAciT8hAd6xi7CnYIg1pG8BQ==", 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-08-15T17:59:15.196 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCWQb5miglkKRAAsM9wIrp5ZtlRKFgGU89dCw==", 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCZQb5msR7DLxAA6uE5hCQk96el7O8p23jaww==", 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCcQb5mW8NfMxAAfzKINNnvlOzcuKyElEvdrg==", 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-08-15T17:59:15.198 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCfQb5m7tbDOhAARcPt7vm+Od/j04RTgRGtqw==", 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQClQb5m71UsJhAAY2vpsr91Ms4feZhJmc9CGg==", 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCoQb5m+Rh1KBAAgryRu/S00b+kD2JdxOIKXQ==", 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCrQb5meR8PLhAAJ/OmtUi8Nze2i8FTNY6vyA==", 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCuQb5mkpgTNBAAOx5A3OVlKXKjs20B8PN7VQ==", 2024-08-15T17:59:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCQQb5mxxgDIhAAQrJS5uiUSE+SwUxjJDxnwQ==", 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-08-15T17:59:15.202 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCQQb5mQQJZJxAArdQLqV7aYscbAcEaJ4sKnA==", 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-08-15T17:59:15.203 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCQQb5mLzKqJBAAZRcwc9MGtYs4OniLCvaUkg==", 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCQQb5m4HirLBAAeXbg/6a1K3aCYTccAbHcwA==", 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-08-15T17:59:15.204 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCQQb5mfPOvARAAFQYxBn4dp6ebVS/dAHjM+w==", 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.205 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-mds", 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCzQb5m3LfkNxAAQX4HEaC/ODzYR2zfGGlCvQ==", 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-mds" 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-mgr", 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCzQb5mFMnkNxAAVnnZ1G1E9f96ywsV+cX61g==", 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-mgr" 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-osd", 2024-08-15T17:59:15.206 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCzQb5mI9vkNxAAEOy6ubS2Ncg28HXGVL09uA==", 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-osd" 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-rbd", 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCzQb5m1OrkNxAAdg0alT8+uA+X8USWoTO+VA==", 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-rbd" 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-08-15T17:59:15.207 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCzQb5mvv3kNxAA6rpcyiz7jfwVeKXIRJMMjA==", 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-rgw", 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCzQb5mkg7lNxAArgYfzyAzcP1nCvpp9pJRmw==", 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-rgw" 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCQQb5m5gIwERAA9K1VvnzPEQJpXRpnB2cy0A==", 2024-08-15T17:59:15.208 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCQQb5mM4CADhAAJobca2kE4NVAbDLtSSirRA==", 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:15.209 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-08-15T17:59:15.210 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T17:59:15.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 -- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6040060890 msgr2=0x7f6040062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6040060890 0x7f6040062d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f60600acad0 tx=0x7f6048001a00 comp rx=0 tx=0).stop 2024-08-15T17:59:15.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 -- 172.21.15.38:0/2824137201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60600686a0 msgr2=0x7f6060158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60600686a0 0x7f6060158390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f605000d890 tx=0x7f6050008040 comp rx=0 tx=0).stop 2024-08-15T17:59:15.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 -- 172.21.15.38:0/2824137201 shutdown_connections 2024-08-15T17:59:15.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6040060890 0x7f6040062d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f60601557b0 0x7f6060152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60601553e0 0x7f6060152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 --2- 172.21.15.38:0/2824137201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60600686a0 0x7f6060158390 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 -- 172.21.15.38:0/2824137201 >> 172.21.15.38:0/2824137201 conn(0x7f606006d800 msgr2=0x7f606006dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 -- 172.21.15.38:0/2824137201 shutdown_connections 2024-08-15T17:59:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.193+0000 7f605f59e700 1 -- 172.21.15.38:0/2824137201 wait complete. 2024-08-15T17:59:15.212 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-08-15T17:59:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.321+0000 7fe448ddf700 1 Processor -- start 2024-08-15T17:59:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.321+0000 7fe448ddf700 1 -- start start 2024-08-15T17:59:15.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe44415a620 0x7fe4440ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4440ae2a0 0x7fe4440a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4440a9160 con 0x7fe4440b0f10 2024-08-15T17:59:15.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4440aff10 con 0x7fe44415a620 2024-08-15T17:59:15.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440b1680 con 0x7fe4440ae2a0 2024-08-15T17:59:15.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe441d9b700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe4440b0f10 0x7fe4440adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35398/0 (socket says 172.21.15.38:35398) 2024-08-15T17:59:15.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44259c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe44415a620 0x7fe4440ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe441d9b700 1 -- 172.21.15.38:0/3151513601 learned_addr learned my addr 172.21.15.38:0/3151513601 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:15.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe442d9d700 1 --2- 172.21.15.38:0/3151513601 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4440ae2a0 0x7fe4440a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44159a700 1 -- 172.21.15.38:0/3151513601 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665428887 0 0) 0x7fe4440a9160 con 0x7fe4440b0f10 2024-08-15T17:59:15.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44159a700 1 -- 172.21.15.38:0/3151513601 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe42c003570 con 0x7fe4440b0f10 2024-08-15T17:59:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44159a700 1 -- 172.21.15.38:0/3151513601 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3216961289 0 0) 0x7fe42c003570 con 0x7fe4440b0f10 2024-08-15T17:59:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44259c700 1 -- 172.21.15.38:0/3151513601 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4440ae2a0 msgr2=0x7fe4440a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44259c700 1 --2- 172.21.15.38:0/3151513601 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4440ae2a0 0x7fe4440a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44259c700 1 -- 172.21.15.38:0/3151513601 >> v1:172.21.15.38:6789/0 conn(0x7fe4440b0f10 legacy=0x7fe4440adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44259c700 1 -- 172.21.15.38:0/3151513601 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4440b1960 con 0x7fe44415a620 2024-08-15T17:59:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe442d9d700 1 --2- 172.21.15.38:0/3151513601 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4440ae2a0 0x7fe4440a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44259c700 1 --2- 172.21.15.38:0/3151513601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe44415a620 0x7fe4440ab170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe4380098e0 tx=0x7fe4380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921a9abe58e297fa server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44159a700 1 -- 172.21.15.38:0/3151513601 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4340017f0 con 0x7fe44415a620 2024-08-15T17:59:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44159a700 1 -- 172.21.15.38:0/3151513601 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe438011420 con 0x7fe44415a620 2024-08-15T17:59:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe44159a700 1 -- 172.21.15.38:0/3151513601 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4380117c0 con 0x7fe44415a620 2024-08-15T17:59:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 -- 172.21.15.38:0/3151513601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe44415a620 msgr2=0x7fe4440ab170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 --2- 172.21.15.38:0/3151513601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe44415a620 0x7fe4440ab170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe4380098e0 tx=0x7fe4380091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 -- 172.21.15.38:0/3151513601 shutdown_connections 2024-08-15T17:59:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 --2- 172.21.15.38:0/3151513601 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4440ae2a0 0x7fe4440a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 --2- 172.21.15.38:0/3151513601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe44415a620 0x7fe4440ab170 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 -- 172.21.15.38:0/3151513601 >> 172.21.15.38:0/3151513601 conn(0x7fe444067810 msgr2=0x7fe444067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 -- 172.21.15.38:0/3151513601 shutdown_connections 2024-08-15T17:59:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.325+0000 7fe448ddf700 1 -- 172.21.15.38:0/3151513601 wait complete. 2024-08-15T17:59:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe448ddf700 1 Processor -- start 2024-08-15T17:59:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe448ddf700 1 -- start start 2024-08-15T17:59:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe448ddf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4440ae2a0 0x7fe444099310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe448ddf700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4440b0f10 0x7fe44414ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe448ddf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe44415a620 0x7fe4440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe448ddf700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe444099850 con 0x7fe44415a620 2024-08-15T17:59:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe448ddf700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe444098e80 con 0x7fe4440ae2a0 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe448ddf700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe444098270 con 0x7fe4440b0f10 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe442d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe44415a620 0x7fe4440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe442d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe44415a620 0x7fe4440693e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35488/0 (socket says 172.21.15.38:35488) 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe442d9d700 1 -- 172.21.15.38:0/1709287980 learned_addr learned my addr 172.21.15.38:0/1709287980 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe44259c700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4440ae2a0 0x7fe444099310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe441d9b700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4440b0f10 0x7fe44414ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe442d9d700 1 -- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4440b0f10 msgr2=0x7fe44414ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe442d9d700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4440b0f10 0x7fe44414ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe442d9d700 1 -- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4440ae2a0 msgr2=0x7fe444099310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe442d9d700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4440ae2a0 0x7fe444099310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe442d9d700 1 -- 172.21.15.38:0/1709287980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c009420 con 0x7fe44415a620 2024-08-15T17:59:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe442d9d700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe44415a620 0x7fe4440693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00ea70 tx=0x7fe43c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe4327fc700 1 -- 172.21.15.38:0/1709287980 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c00eeb0 con 0x7fe44415a620 2024-08-15T17:59:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe4327fc700 1 -- 172.21.15.38:0/1709287980 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c00c850 con 0x7fe44415a620 2024-08-15T17:59:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe448ddf700 1 -- 172.21.15.38:0/1709287980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe438009420 con 0x7fe44415a620 2024-08-15T17:59:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe4327fc700 1 -- 172.21.15.38:0/1709287980 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c00cc30 con 0x7fe44415a620 2024-08-15T17:59:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe448ddf700 1 -- 172.21.15.38:0/1709287980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe444098830 con 0x7fe44415a620 2024-08-15T17:59:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.329+0000 7fe448ddf700 1 -- 172.21.15.38:0/1709287980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe444099010 con 0x7fe44415a620 2024-08-15T17:59:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.333+0000 7fe4327fc700 1 -- 172.21.15.38:0/1709287980 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe43c013470 con 0x7fe44415a620 2024-08-15T17:59:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.333+0000 7fe4327fc700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe424060890 0x7fe424062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.333+0000 7fe4327fc700 1 -- 172.21.15.38:0/1709287980 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe43c07c850 con 0x7fe44415a620 2024-08-15T17:59:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.333+0000 7fe44259c700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe424060890 0x7fe424062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.333+0000 7fe44259c700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe424060890 0x7fe424062d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe4380030c0 tx=0x7fe438016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:15.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.337+0000 7fe4327fc700 1 -- 172.21.15.38:0/1709287980 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe43c04cff0 con 0x7fe44415a620 2024-08-15T17:59:15.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.561+0000 7fe448ddf700 1 -- 172.21.15.38:0/1709287980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe4440645c0 con 0x7fe44415a620 2024-08-15T17:59:15.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.569+0000 7fe4327fc700 1 -- 172.21.15.38:0/1709287980 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe43c050610 con 0x7fe44415a620 2024-08-15T17:59:15.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.569+0000 7fe448ddf700 1 -- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe424060890 msgr2=0x7fe424062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.569+0000 7fe448ddf700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe424060890 0x7fe424062d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe4380030c0 tx=0x7fe438016000 comp rx=0 tx=0).stop 2024-08-15T17:59:15.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.569+0000 7fe448ddf700 1 -- 172.21.15.38:0/1709287980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe44415a620 msgr2=0x7fe4440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.569+0000 7fe448ddf700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe44415a620 0x7fe4440693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00ea70 tx=0x7fe43c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.569+0000 7fe448ddf700 1 -- 172.21.15.38:0/1709287980 shutdown_connections 2024-08-15T17:59:15.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.569+0000 7fe448ddf700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe424060890 0x7fe424062d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.573+0000 7fe448ddf700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe44415a620 0x7fe4440693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.573+0000 7fe448ddf700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4440b0f10 0x7fe44414ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.573+0000 7fe448ddf700 1 --2- 172.21.15.38:0/1709287980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4440ae2a0 0x7fe444099310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.573+0000 7fe448ddf700 1 -- 172.21.15.38:0/1709287980 >> 172.21.15.38:0/1709287980 conn(0x7fe444067810 msgr2=0x7fe4440cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:15.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.573+0000 7fe448ddf700 1 -- 172.21.15.38:0/1709287980 shutdown_connections 2024-08-15T17:59:15.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.573+0000 7fe448ddf700 1 -- 172.21.15.38:0/1709287980 wait complete. 2024-08-15T17:59:15.587 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-08-15T17:59:15.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.685+0000 7f5fbb59e700 1 Processor -- start 2024-08-15T17:59:15.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.685+0000 7f5fbb59e700 1 -- start start 2024-08-15T17:59:15.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbb59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5fbc153320 0x7f5fbc1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbb59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0cd1b0 0x7f5fbc15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbb59e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fbc15dbd0 con 0x7f5fbc15a190 2024-08-15T17:59:15.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbb59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fbc0ce430 con 0x7f5fbc153320 2024-08-15T17:59:15.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbb59e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc15d370 con 0x7f5fbc0cd1b0 2024-08-15T17:59:15.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb9d9b700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5fbc15a190 0x7f5fbc0ccaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35406/0 (socket says 172.21.15.38:35406) 2024-08-15T17:59:15.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb9d9b700 1 -- 172.21.15.38:0/4036503547 learned_addr learned my addr 172.21.15.38:0/4036503547 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:15.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fba59c700 1 --2- 172.21.15.38:0/4036503547 >> v2:172.21.15.104:3300/0 conn(0x7f5fbc153320 0x7f5fbc1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbad9d700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0cd1b0 0x7f5fbc15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809655453 0 0) 0x7f5fbc15dbd0 con 0x7f5fbc15a190 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fa4003570 con 0x7f5fbc15a190 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2747003407 0 0) 0x7f5fa4003570 con 0x7f5fbc15a190 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0cd1b0 msgr2=0x7f5fbc15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0cd1b0 0x7f5fbc15cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 >> v2:172.21.15.104:3300/0 conn(0x7f5fbc153320 msgr2=0x7f5fbc1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 --2- 172.21.15.38:0/4036503547 >> v2:172.21.15.104:3300/0 conn(0x7f5fbc153320 0x7f5fbc1536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fbc15a590 con 0x7f5fbc15a190 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f5fb0001470 con 0x7f5fbc15a190 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 >> v1:172.21.15.38:6789/0 conn(0x7f5fbc15a190 legacy=0x7f5fbc0ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0cd1b0 0x7f5fa4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 0x7f5fa4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fa4004d50 0x7f5fa40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc0ce430 con 0x7f5fa4003fe0 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc15d370 con 0x7f5fa4004d50 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa400ba90 con 0x7f5fbc0cd1b0 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb9d9b700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 0x7f5fa4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fba59c700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0cd1b0 0x7f5fa4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbad9d700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fa4004d50 0x7f5fa40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb9d9b700 1 -- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0cd1b0 msgr2=0x7f5fa4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb9d9b700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0cd1b0 0x7f5fa4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb9d9b700 1 -- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fa4004d50 msgr2=0x7f5fa40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb9d9b700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fa4004d50 0x7f5fa40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb9d9b700 1 -- 172.21.15.38:0/4036503547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa4003570 con 0x7f5fa4003fe0 2024-08-15T17:59:15.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbad9d700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fa4004d50 0x7f5fa40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb9d9b700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 0x7f5fa4004810 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000c3c0 tx=0x7f5fb0001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7fe3f71a9fe8f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000cc30 con 0x7f5fa4003fe0 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb000cdd0 con 0x7f5fa4003fe0 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fb959a700 1 -- 172.21.15.38:0/4036503547 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0005b70 con 0x7f5fa4003fe0 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbb59e700 1 -- 172.21.15.38:0/4036503547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 msgr2=0x7f5fa4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbb59e700 1 --2- 172.21.15.38:0/4036503547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 0x7f5fa4004810 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000c3c0 tx=0x7f5fb0001e90 comp rx=0 tx=0).stop 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbad9d700 1 -- 172.21.15.38:0/4036503547 reap_dead start 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbb59e700 1 -- 172.21.15.38:0/4036503547 shutdown_connections 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbb59e700 1 -- 172.21.15.38:0/4036503547 >> 172.21.15.38:0/4036503547 conn(0x7f5fbc0d00a0 msgr2=0x7f5fbc0d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbb59e700 1 -- 172.21.15.38:0/4036503547 shutdown_connections 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.689+0000 7f5fbb59e700 1 -- 172.21.15.38:0/4036503547 wait complete. 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.693+0000 7f5fbb59e700 1 Processor -- start 2024-08-15T17:59:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.693+0000 7f5fbb59e700 1 -- start start 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.693+0000 7f5fbb59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 0x7f5fbc14a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.693+0000 7f5fbb59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0afb90 0x7f5fbc0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.693+0000 7f5fbb59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fbc0b0540 0x7f5fbc14f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.693+0000 7f5fbb59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc0c57f0 con 0x7f5fa4003fe0 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.693+0000 7f5fbb59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc0c6600 con 0x7f5fbc0b0540 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.693+0000 7f5fbb59e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc0b09b0 con 0x7f5fbc0afb90 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.693+0000 7f5fba59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 0x7f5fbc14a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.693+0000 7f5fba59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 0x7f5fbc14a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35508/0 (socket says 172.21.15.38:35508) 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.693+0000 7f5fba59c700 1 -- 172.21.15.38:0/2469408980 learned_addr learned my addr 172.21.15.38:0/2469408980 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5fb9d9b700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0afb90 0x7f5fbc0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5fbad9d700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fbc0b0540 0x7f5fbc14f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5fba59c700 1 -- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0afb90 msgr2=0x7f5fbc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5fba59c700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0afb90 0x7f5fbc0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5fba59c700 1 -- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fbc0b0540 msgr2=0x7f5fbc14f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5fba59c700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fbc0b0540 0x7f5fbc14f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5fba59c700 1 -- 172.21.15.38:0/2469408980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fbc14a550 con 0x7f5fa4003fe0 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5fba59c700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 0x7f5fbc14a010 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5fac011d70 tx=0x7f5fac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5faa7fc700 1 -- 172.21.15.38:0/2469408980 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00f990 con 0x7f5fa4003fe0 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5faa7fc700 1 -- 172.21.15.38:0/2469408980 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac00faf0 con 0x7f5fa4003fe0 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5fbb59e700 1 -- 172.21.15.38:0/2469408980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fbc0c59f0 con 0x7f5fa4003fe0 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5faa7fc700 1 -- 172.21.15.38:0/2469408980 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00feb0 con 0x7f5fa4003fe0 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5fbb59e700 1 -- 172.21.15.38:0/2469408980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fbc0c5fb0 con 0x7f5fa4003fe0 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5fbb59e700 1 -- 172.21.15.38:0/2469408980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fbc0638f0 con 0x7f5fa4003fe0 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.697+0000 7f5faa7fc700 1 -- 172.21.15.38:0/2469408980 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5fac014650 con 0x7f5fa4003fe0 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.701+0000 7f5faa7fc700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f9c060860 0x7f5f9c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.701+0000 7f5faa7fc700 1 -- 172.21.15.38:0/2469408980 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5fac018030 con 0x7f5fa4003fe0 2024-08-15T17:59:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.701+0000 7f5fb9d9b700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f9c060860 0x7f5f9c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:15.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.701+0000 7f5fb9d9b700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f9c060860 0x7f5f9c062d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000c3c0 tx=0x7f5fb0015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:15.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.709+0000 7f5faa7fc700 1 -- 172.21.15.38:0/2469408980 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5fac0517a0 con 0x7f5fa4003fe0 2024-08-15T17:59:15.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.937+0000 7f5fbb59e700 1 -- 172.21.15.38:0/2469408980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f5fbc0cde20 con 0x7f5fa4003fe0 2024-08-15T17:59:15.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.941+0000 7f5faa7fc700 1 -- 172.21.15.38:0/2469408980 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5fac051330 con 0x7f5fa4003fe0 2024-08-15T17:59:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 -- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f9c060860 msgr2=0x7f5f9c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f9c060860 0x7f5f9c062d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000c3c0 tx=0x7f5fb0015000 comp rx=0 tx=0).stop 2024-08-15T17:59:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 -- 172.21.15.38:0/2469408980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 msgr2=0x7f5fbc14a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 0x7f5fbc14a010 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5fac011d70 tx=0x7f5fac0091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 -- 172.21.15.38:0/2469408980 shutdown_connections 2024-08-15T17:59:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f9c060860 0x7f5f9c062d10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fbc0b0540 0x7f5fbc14f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5fbc0afb90 0x7f5fbc0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 --2- 172.21.15.38:0/2469408980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fa4003fe0 0x7f5fbc14a010 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 -- 172.21.15.38:0/2469408980 >> 172.21.15.38:0/2469408980 conn(0x7f5fbc0d00a0 msgr2=0x7f5fbc06a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:15.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 -- 172.21.15.38:0/2469408980 shutdown_connections 2024-08-15T17:59:15.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:15.945+0000 7f5fbb59e700 1 -- 172.21.15.38:0/2469408980 wait complete. 2024-08-15T17:59:15.960 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-08-15T17:59:16.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.073+0000 7fe3c2e0b700 1 Processor -- start 2024-08-15T17:59:16.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.073+0000 7fe3c2e0b700 1 -- start start 2024-08-15T17:59:16.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.073+0000 7fe3c2e0b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe3bc0b0d20 0x7fe3bc06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.073+0000 7fe3c2e0b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe3bc0b1800 0x7fe3bc141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.073+0000 7fe3c2e0b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3bc141a00 con 0x7fe3bc06e450 2024-08-15T17:59:16.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.073+0000 7fe3c2e0b700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc06e8a0 con 0x7fe3bc0b1800 2024-08-15T17:59:16.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.073+0000 7fe3c2e0b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc140b10 con 0x7fe3bc0b0d20 2024-08-15T17:59:16.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.073+0000 7fe3c1608700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe3bc06e450 0x7fe3bc0b10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35416/0 (socket says 172.21.15.38:35416) 2024-08-15T17:59:16.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.073+0000 7fe3c1608700 1 -- 172.21.15.38:0/3349411509 learned_addr learned my addr 172.21.15.38:0/3349411509 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:16.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.073+0000 7fe3c260a700 1 --2- 172.21.15.38:0/3349411509 >> v2:172.21.15.38:3300/0 conn(0x7fe3bc0b1800 0x7fe3bc141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.073+0000 7fe3c1e09700 1 --2- 172.21.15.38:0/3349411509 >> v2:172.21.15.104:3300/0 conn(0x7fe3bc0b0d20 0x7fe3bc06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c0e07700 1 -- 172.21.15.38:0/3349411509 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033358597 0 0) 0x7fe3bc141a00 con 0x7fe3bc06e450 2024-08-15T17:59:16.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c0e07700 1 -- 172.21.15.38:0/3349411509 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3a8003570 con 0x7fe3bc06e450 2024-08-15T17:59:16.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c260a700 1 -- 172.21.15.38:0/3349411509 >> v2:172.21.15.104:3300/0 conn(0x7fe3bc0b0d20 msgr2=0x7fe3bc06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:16.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c260a700 1 --2- 172.21.15.38:0/3349411509 >> v2:172.21.15.104:3300/0 conn(0x7fe3bc0b0d20 0x7fe3bc06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c260a700 1 -- 172.21.15.38:0/3349411509 >> v1:172.21.15.38:6789/0 conn(0x7fe3bc06e450 legacy=0x7fe3bc0b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c260a700 1 -- 172.21.15.38:0/3349411509 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc140df0 con 0x7fe3bc0b1800 2024-08-15T17:59:16.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c260a700 1 --2- 172.21.15.38:0/3349411509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3bc0b1800 0x7fe3bc141e90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc06bd20 tx=0x7fe3b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1ae839f89d2f4c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:16.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c0e07700 1 -- 172.21.15.38:0/3349411509 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400e070 con 0x7fe3bc0b1800 2024-08-15T17:59:16.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c0e07700 1 -- 172.21.15.38:0/3349411509 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b400bab0 con 0x7fe3bc0b1800 2024-08-15T17:59:16.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c0e07700 1 -- 172.21.15.38:0/3349411509 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400be30 con 0x7fe3bc0b1800 2024-08-15T17:59:16.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/3349411509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3bc0b1800 msgr2=0x7fe3bc141e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 --2- 172.21.15.38:0/3349411509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3bc0b1800 0x7fe3bc141e90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc06bd20 tx=0x7fe3b4009780 comp rx=0 tx=0).stop 2024-08-15T17:59:16.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/3349411509 shutdown_connections 2024-08-15T17:59:16.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 --2- 172.21.15.38:0/3349411509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3bc0b1800 0x7fe3bc141e90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 --2- 172.21.15.38:0/3349411509 >> v2:172.21.15.104:3300/0 conn(0x7fe3bc0b0d20 0x7fe3bc06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/3349411509 >> 172.21.15.38:0/3349411509 conn(0x7fe3bc0678d0 msgr2=0x7fe3bc067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:16.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/3349411509 shutdown_connections 2024-08-15T17:59:16.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/3349411509 wait complete. 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 Processor -- start 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 -- start start 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3bc06e450 0x7fe3bc0a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3bc0b0d20 0x7fe3bc0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe3bc0b1800 0x7fe3bc15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0a69f0 con 0x7fe3bc06e450 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0a6cd0 con 0x7fe3bc0b0d20 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.077+0000 7fe3c2e0b700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0a6fb0 con 0x7fe3bc0b1800 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c1e09700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3bc06e450 0x7fe3bc0a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c1e09700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3bc06e450 0x7fe3bc0a7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35532/0 (socket says 172.21.15.38:35532) 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c1e09700 1 -- 172.21.15.38:0/2383790365 learned_addr learned my addr 172.21.15.38:0/2383790365 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c260a700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe3bc0b1800 0x7fe3bc15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c1608700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3bc0b0d20 0x7fe3bc0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c1e09700 1 -- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe3bc0b1800 msgr2=0x7fe3bc15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c1e09700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe3bc0b1800 0x7fe3bc15daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c1e09700 1 -- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3bc0b0d20 msgr2=0x7fe3bc0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c1e09700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3bc0b0d20 0x7fe3bc0ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c1e09700 1 -- 172.21.15.38:0/2383790365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4009a20 con 0x7fe3bc06e450 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c1608700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3bc0b0d20 0x7fe3bc0ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c1e09700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3bc06e450 0x7fe3bc0a7a90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800ec80 tx=0x7fe3b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3b1ffb700 1 -- 172.21.15.38:0/2383790365 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800c850 con 0x7fe3bc06e450 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3b1ffb700 1 -- 172.21.15.38:0/2383790365 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b800c9b0 con 0x7fe3bc06e450 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/2383790365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc0a7290 con 0x7fe3bc06e450 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3b1ffb700 1 -- 172.21.15.38:0/2383790365 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800cd50 con 0x7fe3bc06e450 2024-08-15T17:59:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/2383790365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc0a7fd0 con 0x7fe3bc06e450 2024-08-15T17:59:16.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.081+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/2383790365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc0643a0 con 0x7fe3bc06e450 2024-08-15T17:59:16.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.085+0000 7fe3b1ffb700 1 -- 172.21.15.38:0/2383790365 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe3b8013680 con 0x7fe3bc06e450 2024-08-15T17:59:16.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.085+0000 7fe3b1ffb700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe3a0060860 0x7fe3a0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.085+0000 7fe3b1ffb700 1 -- 172.21.15.38:0/2383790365 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe3b807ced0 con 0x7fe3bc06e450 2024-08-15T17:59:16.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.085+0000 7fe3c1608700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe3a0060860 0x7fe3a0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.085+0000 7fe3c1608700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe3a0060860 0x7fe3a0062d10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc063170 tx=0x7fe3ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:16.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.093+0000 7fe3b1ffb700 1 -- 172.21.15.38:0/2383790365 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe3b804d5d0 con 0x7fe3bc06e450 2024-08-15T17:59:16.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.317+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/2383790365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fe3bc06f5f0 con 0x7fe3bc06e450 2024-08-15T17:59:16.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.325+0000 7fe3b1ffb700 1 -- 172.21.15.38:0/2383790365 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe3b80170a0 con 0x7fe3bc06e450 2024-08-15T17:59:16.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe3a0060860 msgr2=0x7fe3a0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe3a0060860 0x7fe3a0062d10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc063170 tx=0x7fe3ac009040 comp rx=0 tx=0).stop 2024-08-15T17:59:16.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/2383790365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3bc06e450 msgr2=0x7fe3bc0a7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3bc06e450 0x7fe3bc0a7a90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800ec80 tx=0x7fe3b80091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/2383790365 shutdown_connections 2024-08-15T17:59:16.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe3a0060860 0x7fe3a0062d10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe3bc0b1800 0x7fe3bc15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3bc0b0d20 0x7fe3bc0ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 --2- 172.21.15.38:0/2383790365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3bc06e450 0x7fe3bc0a7a90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/2383790365 >> 172.21.15.38:0/2383790365 conn(0x7fe3bc0678d0 msgr2=0x7fe3bc15bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:16.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/2383790365 shutdown_connections 2024-08-15T17:59:16.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.329+0000 7fe3c2e0b700 1 -- 172.21.15.38:0/2383790365 wait complete. 2024-08-15T17:59:16.347 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-08-15T17:59:16.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f9595d4f700 1 Processor -- start 2024-08-15T17:59:16.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f9595d4f700 1 -- start start 2024-08-15T17:59:16.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f9595d4f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9588050b60 0x7f95880e35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f9595d4f700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9588033f70 0x7f95880343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f9595d4f700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95880337e0 con 0x7f9588050f30 2024-08-15T17:59:16.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f9595d4f700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95880513b0 con 0x7f9588033f70 2024-08-15T17:59:16.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f9595d4f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95880dfd20 con 0x7f9588050b60 2024-08-15T17:59:16.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f959554e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9588033f70 0x7f95880343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f958ffff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9588050f30 0x7f95880df4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35430/0 (socket says 172.21.15.38:35430) 2024-08-15T17:59:16.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f958ffff700 1 -- 172.21.15.38:0/678093119 learned_addr learned my addr 172.21.15.38:0/678093119 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f9594d4d700 1 --2- 172.21.15.38:0/678093119 >> v2:172.21.15.104:3300/0 conn(0x7f9588050b60 0x7f95880e35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f958f7fe700 1 -- 172.21.15.38:0/678093119 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265718614 0 0) 0x7f95880337e0 con 0x7f9588050f30 2024-08-15T17:59:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.457+0000 7f958f7fe700 1 -- 172.21.15.38:0/678093119 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9580003570 con 0x7f9588050f30 2024-08-15T17:59:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f959554e700 1 -- 172.21.15.38:0/678093119 >> v2:172.21.15.104:3300/0 conn(0x7f9588050b60 msgr2=0x7f95880e35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f959554e700 1 --2- 172.21.15.38:0/678093119 >> v2:172.21.15.104:3300/0 conn(0x7f9588050b60 0x7f95880e35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f959554e700 1 -- 172.21.15.38:0/678093119 >> v1:172.21.15.38:6789/0 conn(0x7f9588050f30 legacy=0x7f95880df4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f959554e700 1 -- 172.21.15.38:0/678093119 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880e0000 con 0x7f9588033f70 2024-08-15T17:59:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f959554e700 1 --2- 172.21.15.38:0/678093119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9588033f70 0x7f95880343f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9590065b20 tx=0x7f9590065310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576db3dde8c1f873 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f958f7fe700 1 -- 172.21.15.38:0/678093119 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959004c190 con 0x7f9588033f70 2024-08-15T17:59:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f958f7fe700 1 -- 172.21.15.38:0/678093119 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9590029ba0 con 0x7f9588033f70 2024-08-15T17:59:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f958f7fe700 1 -- 172.21.15.38:0/678093119 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590064870 con 0x7f9588033f70 2024-08-15T17:59:16.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f9595d4f700 1 -- 172.21.15.38:0/678093119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9588033f70 msgr2=0x7f95880343f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f9595d4f700 1 --2- 172.21.15.38:0/678093119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9588033f70 0x7f95880343f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9590065b20 tx=0x7f9590065310 comp rx=0 tx=0).stop 2024-08-15T17:59:16.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f9595d4f700 1 -- 172.21.15.38:0/678093119 shutdown_connections 2024-08-15T17:59:16.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f9595d4f700 1 --2- 172.21.15.38:0/678093119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9588033f70 0x7f95880343f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f9595d4f700 1 --2- 172.21.15.38:0/678093119 >> v2:172.21.15.104:3300/0 conn(0x7f9588050b60 0x7f95880e35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f9595d4f700 1 -- 172.21.15.38:0/678093119 >> 172.21.15.38:0/678093119 conn(0x7f958800a030 msgr2=0x7f958800a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:16.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f9595d4f700 1 -- 172.21.15.38:0/678093119 shutdown_connections 2024-08-15T17:59:16.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f9595d4f700 1 -- 172.21.15.38:0/678093119 wait complete. 2024-08-15T17:59:16.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f9595d4f700 1 Processor -- start 2024-08-15T17:59:16.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.461+0000 7f9595d4f700 1 -- start start 2024-08-15T17:59:16.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f9595d4f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9588033f70 0x7f9588051ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f9595d4f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9588050b60 0x7f95880da050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f9595d4f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9588050f30 0x7f95880da590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f9595d4f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958803a740 con 0x7f9588050f30 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f9595d4f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95880ee430 con 0x7f9588050b60 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f9595d4f700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9588052400 con 0x7f9588033f70 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f959554e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9588050f30 0x7f95880da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f959554e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9588050f30 0x7f95880da590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35550/0 (socket says 172.21.15.38:35550) 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f959554e700 1 -- 172.21.15.38:0/1942666732 learned_addr learned my addr 172.21.15.38:0/1942666732 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f9594d4d700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9588033f70 0x7f9588051ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f958ffff700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9588050b60 0x7f95880da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f959554e700 1 -- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9588033f70 msgr2=0x7f9588051ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f959554e700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9588033f70 0x7f9588051ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f959554e700 1 -- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9588050b60 msgr2=0x7f95880da050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f959554e700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9588050b60 0x7f95880da050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f959554e700 1 -- 172.21.15.38:0/1942666732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959004c770 con 0x7f9588050f30 2024-08-15T17:59:16.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f958ffff700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9588050b60 0x7f95880da050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:16.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f9594d4d700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9588033f70 0x7f9588051ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:16.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f959554e700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9588050f30 0x7f95880da590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9590066b10 tx=0x7f959006c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:16.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f958cff9700 1 -- 172.21.15.38:0/1942666732 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590070af0 con 0x7f9588050f30 2024-08-15T17:59:16.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f958cff9700 1 -- 172.21.15.38:0/1942666732 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9590070c50 con 0x7f9588050f30 2024-08-15T17:59:16.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f9595d4f700 1 -- 172.21.15.38:0/1942666732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95880edb00 con 0x7f9588050f30 2024-08-15T17:59:16.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f958cff9700 1 -- 172.21.15.38:0/1942666732 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590070dc0 con 0x7f9588050f30 2024-08-15T17:59:16.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f9595d4f700 1 -- 172.21.15.38:0/1942666732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95880ee0c0 con 0x7f9588050f30 2024-08-15T17:59:16.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f9595d4f700 1 -- 172.21.15.38:0/1942666732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95880ee5c0 con 0x7f9588050f30 2024-08-15T17:59:16.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.465+0000 7f958cff9700 1 -- 172.21.15.38:0/1942666732 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9590098020 con 0x7f9588050f30 2024-08-15T17:59:16.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.469+0000 7f958cff9700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9574060860 0x7f9574062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.469+0000 7f958cff9700 1 -- 172.21.15.38:0/1942666732 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f95900f4f50 con 0x7f9588050f30 2024-08-15T17:59:16.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.469+0000 7f9594d4d700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9574060860 0x7f9574062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.469+0000 7f9594d4d700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9574060860 0x7f9574062d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9584009d80 tx=0x7f9584008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:16.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.477+0000 7f958cff9700 1 -- 172.21.15.38:0/1942666732 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f95900c56f0 con 0x7f9588050f30 2024-08-15T17:59:16.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.701+0000 7f9595d4f700 1 -- 172.21.15.38:0/1942666732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f95880053b0 con 0x7f9588050f30 2024-08-15T17:59:16.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.709+0000 7f958cff9700 1 -- 172.21.15.38:0/1942666732 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f95900c8d10 con 0x7f9588050f30 2024-08-15T17:59:16.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 -- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9574060860 msgr2=0x7f9574062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9574060860 0x7f9574062d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9584009d80 tx=0x7f9584008040 comp rx=0 tx=0).stop 2024-08-15T17:59:16.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 -- 172.21.15.38:0/1942666732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9588050f30 msgr2=0x7f95880da590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9588050f30 0x7f95880da590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9590066b10 tx=0x7f959006c070 comp rx=0 tx=0).stop 2024-08-15T17:59:16.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 -- 172.21.15.38:0/1942666732 shutdown_connections 2024-08-15T17:59:16.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9574060860 0x7f9574062d10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9588050f30 0x7f95880da590 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9588050b60 0x7f95880da050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 --2- 172.21.15.38:0/1942666732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9588033f70 0x7f9588051ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 -- 172.21.15.38:0/1942666732 >> 172.21.15.38:0/1942666732 conn(0x7f958800a030 msgr2=0x7f958800a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:16.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 -- 172.21.15.38:0/1942666732 shutdown_connections 2024-08-15T17:59:16.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.713+0000 7f9595d4f700 1 -- 172.21.15.38:0/1942666732 wait complete. 2024-08-15T17:59:16.729 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-08-15T17:59:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3366693700 1 Processor -- start 2024-08-15T17:59:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3366693700 1 -- start start 2024-08-15T17:59:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3366693700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33600b0350 0x7f33600ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3366693700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33600af2b0 con 0x7f3360155d20 2024-08-15T17:59:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3366693700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33600af590 con 0x7f336006df10 2024-08-15T17:59:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3366693700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33600b07a0 con 0x7f33600b0350 2024-08-15T17:59:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3365691700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f3360155d20 0x7f33600aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59130/0 (socket says 172.21.15.38:59130) 2024-08-15T17:59:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3365691700 1 -- 172.21.15.38:0/4172756074 learned_addr learned my addr 172.21.15.38:0/4172756074 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3364e90700 1 --2- 172.21.15.38:0/4172756074 >> v2:172.21.15.38:3300/0 conn(0x7f33600b0350 0x7f33600ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217675905 0 0) 0x7f33600af2b0 con 0x7f3360155d20 2024-08-15T17:59:16.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f334c003570 con 0x7f3360155d20 2024-08-15T17:59:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1247344106 0 0) 0x7f334c003570 con 0x7f3360155d20 2024-08-15T17:59:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 >> v2:172.21.15.38:3300/0 conn(0x7f33600b0350 msgr2=0x7f33600ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 --2- 172.21.15.38:0/4172756074 >> v2:172.21.15.38:3300/0 conn(0x7f33600b0350 0x7f33600ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 >> v1:172.21.15.104:6789/0 conn(0x7f336006df10 legacy=0x7f336006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33600af870 con 0x7f3360155d20 2024-08-15T17:59:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f33580017f0 con 0x7f3360155d20 2024-08-15T17:59:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 >> v1:172.21.15.38:6789/0 conn(0x7f3360155d20 legacy=0x7f33600aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f334c003a60 0x7f334c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f334c004810 0x7f334c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f334c0051c0 0x7f334c00dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c00e2b0 con 0x7f334c0051c0 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c00e420 con 0x7f334c004810 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c00e5e0 con 0x7f334c003a60 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3365e92700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f334c0051c0 0x7f334c00dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3364e90700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f334c004810 0x7f334c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.837+0000 7f3365691700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f334c003a60 0x7f334c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3365e92700 1 -- 172.21.15.38:0/4172756074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f334c003a60 msgr2=0x7f334c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3365e92700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f334c003a60 0x7f334c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3365e92700 1 -- 172.21.15.38:0/4172756074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f334c004810 msgr2=0x7f334c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3365e92700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f334c004810 0x7f334c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3365e92700 1 -- 172.21.15.38:0/4172756074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c003570 con 0x7f334c0051c0 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3364e90700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f334c004810 0x7f334c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3365e92700 1 -- 172.21.15.38:0/4172756074 reap_dead start 2024-08-15T17:59:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3365e92700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f334c0051c0 0x7f334c00dc70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f335c0018b0 tx=0x7f335c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4a20d6e63df008 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c012420 con 0x7f334c0051c0 2024-08-15T17:59:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335c0125a0 con 0x7f334c0051c0 2024-08-15T17:59:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3357fff700 1 -- 172.21.15.38:0/4172756074 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c012960 con 0x7f334c0051c0 2024-08-15T17:59:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3366693700 1 -- 172.21.15.38:0/4172756074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f334c0051c0 msgr2=0x7f334c00dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3366693700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f334c0051c0 0x7f334c00dc70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f335c0018b0 tx=0x7f335c007d30 comp rx=0 tx=0).stop 2024-08-15T17:59:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3366693700 1 -- 172.21.15.38:0/4172756074 shutdown_connections 2024-08-15T17:59:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3366693700 1 --2- 172.21.15.38:0/4172756074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f334c0051c0 0x7f334c00dc70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3366693700 1 -- 172.21.15.38:0/4172756074 >> 172.21.15.38:0/4172756074 conn(0x7f33600678d0 msgr2=0x7f3360067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3366693700 1 -- 172.21.15.38:0/4172756074 shutdown_connections 2024-08-15T17:59:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3366693700 1 -- 172.21.15.38:0/4172756074 wait complete. 2024-08-15T17:59:16.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3366693700 1 Processor -- start 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.841+0000 7f3366693700 1 -- start start 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3366693700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f334c0051c0 0x7f33601588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3366693700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3360148530 0x7f3360158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3366693700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3360153d00 0x7f3360154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3366693700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3360148d90 con 0x7f3360148530 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3366693700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33601567a0 con 0x7f334c0051c0 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3366693700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33601546b0 con 0x7f3360153d00 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3364e90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3360148530 0x7f3360158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3365691700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f334c0051c0 0x7f33601588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3364e90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3360148530 0x7f3360158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41600/0 (socket says 172.21.15.38:41600) 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3365691700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f334c0051c0 0x7f33601588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:55632/0 (socket says 172.21.15.38:55632) 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3364e90700 1 -- 172.21.15.38:0/3631844404 learned_addr learned my addr 172.21.15.38:0/3631844404 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3365e92700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3360153d00 0x7f3360154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3364e90700 1 -- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3360153d00 msgr2=0x7f3360154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3364e90700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3360153d00 0x7f3360154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3364e90700 1 -- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f334c0051c0 msgr2=0x7f33601588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:16.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3364e90700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f334c0051c0 0x7f33601588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:16.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3364e90700 1 -- 172.21.15.38:0/3631844404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3360156190 con 0x7f3360148530 2024-08-15T17:59:16.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3364e90700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3360148530 0x7f3360158df0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3350011730 tx=0x7f3350008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:16.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f33557fa700 1 -- 172.21.15.38:0/3631844404 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350015740 con 0x7f3360148530 2024-08-15T17:59:16.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f33557fa700 1 -- 172.21.15.38:0/3631844404 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33500158a0 con 0x7f3360148530 2024-08-15T17:59:16.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3366693700 1 -- 172.21.15.38:0/3631844404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3360154930 con 0x7f3360148530 2024-08-15T17:59:16.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f33557fa700 1 -- 172.21.15.38:0/3631844404 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350015c40 con 0x7f3360148530 2024-08-15T17:59:16.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.845+0000 7f3366693700 1 -- 172.21.15.38:0/3631844404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3360154c10 con 0x7f3360148530 2024-08-15T17:59:16.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.849+0000 7f3366693700 1 -- 172.21.15.38:0/3631844404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3360156950 con 0x7f3360148530 2024-08-15T17:59:16.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.849+0000 7f33557fa700 1 -- 172.21.15.38:0/3631844404 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f335001e450 con 0x7f3360148530 2024-08-15T17:59:16.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.849+0000 7f33557fa700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3344060860 0x7f3344062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:16.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.849+0000 7f33557fa700 1 -- 172.21.15.38:0/3631844404 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f335007ce40 con 0x7f3360148530 2024-08-15T17:59:16.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.849+0000 7f3365691700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3344060860 0x7f3344062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:16.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.853+0000 7f3365691700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3344060860 0x7f3344062d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3358001190 tx=0x7f335800c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:16.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:16.857+0000 7f33557fa700 1 -- 172.21.15.38:0/3631844404 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f335004d5c0 con 0x7f3360148530 2024-08-15T17:59:17.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.081+0000 7f3366693700 1 -- 172.21.15.38:0/3631844404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f3360156910 con 0x7f3360148530 2024-08-15T17:59:17.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f33557fa700 1 -- 172.21.15.38:0/3631844404 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f33500190a0 con 0x7f3360148530 2024-08-15T17:59:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 -- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3344060860 msgr2=0x7f3344062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3344060860 0x7f3344062d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3358001190 tx=0x7f335800c040 comp rx=0 tx=0).stop 2024-08-15T17:59:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 -- 172.21.15.38:0/3631844404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3360148530 msgr2=0x7f3360158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3360148530 0x7f3360158df0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3350011730 tx=0x7f3350008040 comp rx=0 tx=0).stop 2024-08-15T17:59:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 -- 172.21.15.38:0/3631844404 shutdown_connections 2024-08-15T17:59:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3344060860 0x7f3344062d10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3360153d00 0x7f3360154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3360148530 0x7f3360158df0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 --2- 172.21.15.38:0/3631844404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f334c0051c0 0x7f33601588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 -- 172.21.15.38:0/3631844404 >> 172.21.15.38:0/3631844404 conn(0x7f33600678d0 msgr2=0x7f3360069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:17.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 -- 172.21.15.38:0/3631844404 shutdown_connections 2024-08-15T17:59:17.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.089+0000 7f3366693700 1 -- 172.21.15.38:0/3631844404 wait complete. 2024-08-15T17:59:17.106 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-08-15T17:59:17.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.213+0000 7f32c67b3700 1 Processor -- start 2024-08-15T17:59:17.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.213+0000 7f32c67b3700 1 -- start start 2024-08-15T17:59:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c0159340 0x7f32c0159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32c0159c50 0x7f32c00b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f32c00b4160 0x7f32c00b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c00b45e0 con 0x7f32c0159c50 2024-08-15T17:59:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c00b1cf0 con 0x7f32c00b4160 2024-08-15T17:59:17.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c00b60d0 con 0x7f32c0159340 2024-08-15T17:59:17.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32bf7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32c0159c50 0x7f32c00b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32bf7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32c0159c50 0x7f32c00b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41602/0 (socket says 172.21.15.38:41602) 2024-08-15T17:59:17.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32bf7fe700 1 -- 172.21.15.38:0/3402662940 learned_addr learned my addr 172.21.15.38:0/3402662940 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:17.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32bffff700 1 --2- 172.21.15.38:0/3402662940 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c0159340 0x7f32c0159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c4d50700 1 --2- 172.21.15.38:0/3402662940 >> v2:172.21.15.104:3300/0 conn(0x7f32c00b4160 0x7f32c00b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32bf7fe700 1 -- 172.21.15.38:0/3402662940 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c0159340 msgr2=0x7f32c0159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32bf7fe700 1 --2- 172.21.15.38:0/3402662940 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c0159340 0x7f32c0159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32bf7fe700 1 -- 172.21.15.38:0/3402662940 >> v2:172.21.15.104:3300/0 conn(0x7f32c00b4160 msgr2=0x7f32c00b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32bf7fe700 1 --2- 172.21.15.38:0/3402662940 >> v2:172.21.15.104:3300/0 conn(0x7f32c00b4160 0x7f32c00b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32bf7fe700 1 -- 172.21.15.38:0/3402662940 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c00b10e0 con 0x7f32c0159c50 2024-08-15T17:59:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c4d50700 1 --2- 172.21.15.38:0/3402662940 >> v2:172.21.15.104:3300/0 conn(0x7f32c00b4160 0x7f32c00b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32bf7fe700 1 --2- 172.21.15.38:0/3402662940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32c0159c50 0x7f32c00b3c20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f32c0089110 tx=0x7f32a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd9343026810983 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32beffd700 1 -- 172.21.15.38:0/3402662940 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8009e70 con 0x7f32c0159c50 2024-08-15T17:59:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32beffd700 1 -- 172.21.15.38:0/3402662940 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a800b860 con 0x7f32c0159c50 2024-08-15T17:59:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32beffd700 1 -- 172.21.15.38:0/3402662940 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a800bc00 con 0x7f32c0159c50 2024-08-15T17:59:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 -- 172.21.15.38:0/3402662940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32c0159c50 msgr2=0x7f32c00b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 --2- 172.21.15.38:0/3402662940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32c0159c50 0x7f32c00b3c20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f32c0089110 tx=0x7f32a8009250 comp rx=0 tx=0).stop 2024-08-15T17:59:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 -- 172.21.15.38:0/3402662940 shutdown_connections 2024-08-15T17:59:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 --2- 172.21.15.38:0/3402662940 >> v2:172.21.15.104:3300/0 conn(0x7f32c00b4160 0x7f32c00b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 --2- 172.21.15.38:0/3402662940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32c0159c50 0x7f32c00b3c20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 --2- 172.21.15.38:0/3402662940 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c0159340 0x7f32c0159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 -- 172.21.15.38:0/3402662940 >> 172.21.15.38:0/3402662940 conn(0x7f32c00994f0 msgr2=0x7f32c00675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 -- 172.21.15.38:0/3402662940 shutdown_connections 2024-08-15T17:59:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.217+0000 7f32c67b3700 1 -- 172.21.15.38:0/3402662940 wait complete. 2024-08-15T17:59:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32c67b3700 1 Processor -- start 2024-08-15T17:59:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32c67b3700 1 -- start start 2024-08-15T17:59:17.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32c67b3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32c00b4160 0x7f32c014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32c67b3700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c0159340 0x7f32c0152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32c67b3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0159c50 0x7f32c0151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32c67b3700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c0155590 con 0x7f32c00b4160 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32c67b3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c014eca0 con 0x7f32c0159c50 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32c67b3700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c0154980 con 0x7f32c0159340 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32bffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32c00b4160 0x7f32c014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32bffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32c00b4160 0x7f32c014e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41614/0 (socket says 172.21.15.38:41614) 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32bffff700 1 -- 172.21.15.38:0/1692518786 learned_addr learned my addr 172.21.15.38:0/1692518786 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32bf7fe700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c0159340 0x7f32c0152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.221+0000 7f32c4d50700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0159c50 0x7f32c0151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32bffff700 1 -- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c0159340 msgr2=0x7f32c0152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32bffff700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c0159340 0x7f32c0152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32bffff700 1 -- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0159c50 msgr2=0x7f32c0151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32bffff700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0159c50 0x7f32c0151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32bffff700 1 -- 172.21.15.38:0/1692518786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a80094f0 con 0x7f32c00b4160 2024-08-15T17:59:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32c4d50700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0159c50 0x7f32c0151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32bffff700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32c00b4160 0x7f32c014e760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f32b000ec80 tx=0x7f32b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32b7fff700 1 -- 172.21.15.38:0/1692518786 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b000c850 con 0x7f32c00b4160 2024-08-15T17:59:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32b7fff700 1 -- 172.21.15.38:0/1692518786 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b000c9b0 con 0x7f32c00b4160 2024-08-15T17:59:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32c67b3700 1 -- 172.21.15.38:0/1692518786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c0154c60 con 0x7f32c00b4160 2024-08-15T17:59:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32b7fff700 1 -- 172.21.15.38:0/1692518786 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b000cd50 con 0x7f32c00b4160 2024-08-15T17:59:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32c67b3700 1 -- 172.21.15.38:0/1692518786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c0155220 con 0x7f32c00b4160 2024-08-15T17:59:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.225+0000 7f32c67b3700 1 -- 172.21.15.38:0/1692518786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c00645c0 con 0x7f32c00b4160 2024-08-15T17:59:17.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.229+0000 7f32b7fff700 1 -- 172.21.15.38:0/1692518786 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f32b0013680 con 0x7f32c00b4160 2024-08-15T17:59:17.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.229+0000 7f32b7fff700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f32a0060860 0x7f32a0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.229+0000 7f32b7fff700 1 -- 172.21.15.38:0/1692518786 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f32b007d350 con 0x7f32c00b4160 2024-08-15T17:59:17.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.229+0000 7f32bf7fe700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f32a0060860 0x7f32a0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.229+0000 7f32bf7fe700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f32a0060860 0x7f32a0062d10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f32c0089110 tx=0x7f32a8017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:17.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.237+0000 7f32b7fff700 1 -- 172.21.15.38:0/1692518786 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f32b004daf0 con 0x7f32c00b4160 2024-08-15T17:59:17.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.461+0000 7f32c67b3700 1 -- 172.21.15.38:0/1692518786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f32c00aaa40 con 0x7f32c00b4160 2024-08-15T17:59:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.465+0000 7f32b7fff700 1 -- 172.21.15.38:0/1692518786 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f32b0051110 con 0x7f32c00b4160 2024-08-15T17:59:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 -- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f32a0060860 msgr2=0x7f32a0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f32a0060860 0x7f32a0062d10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f32c0089110 tx=0x7f32a8017000 comp rx=0 tx=0).stop 2024-08-15T17:59:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 -- 172.21.15.38:0/1692518786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32c00b4160 msgr2=0x7f32c014e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32c00b4160 0x7f32c014e760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f32b000ec80 tx=0x7f32b00091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 -- 172.21.15.38:0/1692518786 shutdown_connections 2024-08-15T17:59:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f32a0060860 0x7f32a0062d10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c0159c50 0x7f32c0151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c0159340 0x7f32c0152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 --2- 172.21.15.38:0/1692518786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32c00b4160 0x7f32c014e760 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 -- 172.21.15.38:0/1692518786 >> 172.21.15.38:0/1692518786 conn(0x7f32c00994f0 msgr2=0x7f32c015aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 -- 172.21.15.38:0/1692518786 shutdown_connections 2024-08-15T17:59:17.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.469+0000 7f32c67b3700 1 -- 172.21.15.38:0/1692518786 wait complete. 2024-08-15T17:59:17.486 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:59:17.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.601+0000 7f453a0a5700 1 Processor -- start 2024-08-15T17:59:17.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.601+0000 7f453a0a5700 1 -- start start 2024-08-15T17:59:17.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f4534155e50 0x7f45340a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4534156220 0x7f45340aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45340afc10 con 0x7f453406ebe0 2024-08-15T17:59:17.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45340b10b0 con 0x7f4534155e50 2024-08-15T17:59:17.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45340aed20 con 0x7f4534156220 2024-08-15T17:59:17.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f4533fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f453406ebe0 0x7f4534159bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59140/0 (socket says 172.21.15.38:59140) 2024-08-15T17:59:17.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f4533fff700 1 -- 172.21.15.38:0/1274417031 learned_addr learned my addr 172.21.15.38:0/1274417031 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:17.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f45337fe700 1 --2- 172.21.15.38:0/1274417031 >> v2:172.21.15.104:3300/0 conn(0x7f4534155e50 0x7f45340a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f452bfff700 1 --2- 172.21.15.38:0/1274417031 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4534156220 0x7f45340aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f4532ffd700 1 -- 172.21.15.38:0/1274417031 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492559707 0 0) 0x7f45340afc10 con 0x7f453406ebe0 2024-08-15T17:59:17.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f4532ffd700 1 -- 172.21.15.38:0/1274417031 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4520003570 con 0x7f453406ebe0 2024-08-15T17:59:17.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f45337fe700 1 -- 172.21.15.38:0/1274417031 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4534156220 msgr2=0x7f45340aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:17.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f45337fe700 1 --2- 172.21.15.38:0/1274417031 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4534156220 0x7f45340aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f45337fe700 1 -- 172.21.15.38:0/1274417031 >> v1:172.21.15.38:6789/0 conn(0x7f453406ebe0 legacy=0x7f4534159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f45337fe700 1 -- 172.21.15.38:0/1274417031 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340af000 con 0x7f4534155e50 2024-08-15T17:59:17.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f45337fe700 1 --2- 172.21.15.38:0/1274417031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4534155e50 0x7f45340a9ee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45240098e0 tx=0x7f45240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da773509269ca1d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:17.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f4532ffd700 1 -- 172.21.15.38:0/1274417031 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524011630 con 0x7f4534155e50 2024-08-15T17:59:17.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f4532ffd700 1 -- 172.21.15.38:0/1274417031 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4524011790 con 0x7f4534155e50 2024-08-15T17:59:17.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f4532ffd700 1 -- 172.21.15.38:0/1274417031 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524011b10 con 0x7f4534155e50 2024-08-15T17:59:17.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 -- 172.21.15.38:0/1274417031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4534155e50 msgr2=0x7f45340a9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 --2- 172.21.15.38:0/1274417031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4534155e50 0x7f45340a9ee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45240098e0 tx=0x7f45240091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 -- 172.21.15.38:0/1274417031 shutdown_connections 2024-08-15T17:59:17.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 --2- 172.21.15.38:0/1274417031 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4534156220 0x7f45340aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 --2- 172.21.15.38:0/1274417031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4534155e50 0x7f45340a9ee0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 -- 172.21.15.38:0/1274417031 >> 172.21.15.38:0/1274417031 conn(0x7f4534066e00 msgr2=0x7f4534067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:17.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 -- 172.21.15.38:0/1274417031 shutdown_connections 2024-08-15T17:59:17.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.605+0000 7f453a0a5700 1 -- 172.21.15.38:0/1274417031 wait complete. 2024-08-15T17:59:17.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f453a0a5700 1 Processor -- start 2024-08-15T17:59:17.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f453a0a5700 1 -- start start 2024-08-15T17:59:17.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f453a0a5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f453406ebe0 0x7f453415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f453a0a5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4534155e50 0x7f4534152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f453a0a5700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4534156220 0x7f4534098c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f453a0a5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45341530c0 con 0x7f4534155e50 2024-08-15T17:59:17.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f453a0a5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4534098770 con 0x7f453406ebe0 2024-08-15T17:59:17.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f453a0a5700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4534097b60 con 0x7f4534156220 2024-08-15T17:59:17.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f452bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4534155e50 0x7f4534152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f45337fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f453406ebe0 0x7f453415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f452bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4534155e50 0x7f4534152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41620/0 (socket says 172.21.15.38:41620) 2024-08-15T17:59:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f452bfff700 1 -- 172.21.15.38:0/3787172086 learned_addr learned my addr 172.21.15.38:0/3787172086 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f4533fff700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4534156220 0x7f4534098c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f452bfff700 1 -- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4534156220 msgr2=0x7f4534098c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f452bfff700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4534156220 0x7f4534098c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f452bfff700 1 -- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f453406ebe0 msgr2=0x7f453415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f452bfff700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f453406ebe0 0x7f453415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f452bfff700 1 -- 172.21.15.38:0/3787172086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4524009420 con 0x7f4534155e50 2024-08-15T17:59:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f4533fff700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4534156220 0x7f4534098c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f452bfff700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4534155e50 0x7f4534152b80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f451c00e900 tx=0x7f451c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f452b7fe700 1 -- 172.21.15.38:0/3787172086 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c013680 con 0x7f4534155e50 2024-08-15T17:59:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f452b7fe700 1 -- 172.21.15.38:0/3787172086 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c0137e0 con 0x7f4534155e50 2024-08-15T17:59:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.609+0000 7f453a0a5700 1 -- 172.21.15.38:0/3787172086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4534097e40 con 0x7f4534155e50 2024-08-15T17:59:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.613+0000 7f452b7fe700 1 -- 172.21.15.38:0/3787172086 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c013ba0 con 0x7f4534155e50 2024-08-15T17:59:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.613+0000 7f453a0a5700 1 -- 172.21.15.38:0/3787172086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4534098400 con 0x7f4534155e50 2024-08-15T17:59:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.613+0000 7f453a0a5700 1 -- 172.21.15.38:0/3787172086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45340645c0 con 0x7f4534155e50 2024-08-15T17:59:17.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.613+0000 7f452b7fe700 1 -- 172.21.15.38:0/3787172086 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f451c00c460 con 0x7f4534155e50 2024-08-15T17:59:17.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.613+0000 7f452b7fe700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4514060860 0x7f4514062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.613+0000 7f452b7fe700 1 -- 172.21.15.38:0/3787172086 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f451c07c9e0 con 0x7f4534155e50 2024-08-15T17:59:17.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.613+0000 7f45337fe700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4514060860 0x7f4514062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.617+0000 7f45337fe700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4514060860 0x7f4514062d10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f45240091e0 tx=0x7f4524017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:17.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.621+0000 7f452b7fe700 1 -- 172.21.15.38:0/3787172086 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f451c04d180 con 0x7f4534155e50 2024-08-15T17:59:17.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.789+0000 7f453a0a5700 1 -- 172.21.15.38:0/3787172086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45340ae670 con 0x7f4534155e50 2024-08-15T17:59:17.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.793+0000 7f452b7fe700 1 -- 172.21.15.38:0/3787172086 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+12134 (secure 0 0 0) 0x7f451c0507a0 con 0x7f4534155e50 2024-08-15T17:59:17.795 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:17.795 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:59:14.836376+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:59:17.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 -- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4514060860 msgr2=0x7f4514062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4514060860 0x7f4514062d10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f45240091e0 tx=0x7f4524017000 comp rx=0 tx=0).stop 2024-08-15T17:59:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 -- 172.21.15.38:0/3787172086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4534155e50 msgr2=0x7f4534152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4534155e50 0x7f4534152b80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f451c00e900 tx=0x7f451c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 -- 172.21.15.38:0/3787172086 shutdown_connections 2024-08-15T17:59:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4514060860 0x7f4514062d10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4534156220 0x7f4534098c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4534155e50 0x7f4534152b80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 --2- 172.21.15.38:0/3787172086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f453406ebe0 0x7f453415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 -- 172.21.15.38:0/3787172086 >> 172.21.15.38:0/3787172086 conn(0x7f4534066e00 msgr2=0x7f4534153aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 -- 172.21.15.38:0/3787172086 shutdown_connections 2024-08-15T17:59:17.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.797+0000 7f453a0a5700 1 -- 172.21.15.38:0/3787172086 wait complete. 2024-08-15T17:59:17.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-15T17:59:17.816 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:59:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.925+0000 7f381dd2e700 1 Processor -- start 2024-08-15T17:59:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.925+0000 7f381dd2e700 1 -- start start 2024-08-15T17:59:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381dd2e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3818067030 0x7f3818067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381dd2e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38180cf670 con 0x7f38180cfb00 2024-08-15T17:59:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381dd2e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38180cea60 con 0x7f3818158cc0 2024-08-15T17:59:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381dd2e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38180ced40 con 0x7f3818067030 2024-08-15T17:59:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f3817fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f38180cfb00 0x7f38180cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59150/0 (socket says 172.21.15.38:59150) 2024-08-15T17:59:17.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f3817fff700 1 -- 172.21.15.38:0/1607446032 learned_addr learned my addr 172.21.15.38:0/1607446032 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:17.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381cd2c700 1 --2- 172.21.15.38:0/1607446032 >> v2:172.21.15.38:3300/0 conn(0x7f3818067030 0x7f3818067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f38177fe700 1 -- 172.21.15.38:0/1607446032 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433043032 0 0) 0x7f38180cf670 con 0x7f38180cfb00 2024-08-15T17:59:17.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f38177fe700 1 -- 172.21.15.38:0/1607446032 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3804003570 con 0x7f38180cfb00 2024-08-15T17:59:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f38177fe700 1 -- 172.21.15.38:0/1607446032 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272301006 0 0) 0x7f38180cea60 con 0x7f3818158cc0 2024-08-15T17:59:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f38177fe700 1 -- 172.21.15.38:0/1607446032 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38180cf670 con 0x7f3818158cc0 2024-08-15T17:59:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381cd2c700 1 -- 172.21.15.38:0/1607446032 >> v1:172.21.15.104:6789/0 conn(0x7f3818158cc0 legacy=0x7f38181590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381cd2c700 1 -- 172.21.15.38:0/1607446032 >> v1:172.21.15.38:6789/0 conn(0x7f38180cfb00 legacy=0x7f38180cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f38177fe700 1 -- 172.21.15.38:0/1607446032 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1226230703 0 0) 0x7f3804003570 con 0x7f38180cfb00 2024-08-15T17:59:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381cd2c700 1 -- 172.21.15.38:0/1607446032 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38180cf020 con 0x7f3818067030 2024-08-15T17:59:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381cd2c700 1 --2- 172.21.15.38:0/1607446032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3818067030 0x7f3818067400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f38080098e0 tx=0x7f38080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d255429071290 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f38177fe700 1 -- 172.21.15.38:0/1607446032 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800001470 con 0x7f3818067030 2024-08-15T17:59:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f38177fe700 1 -- 172.21.15.38:0/1607446032 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3808011420 con 0x7f3818067030 2024-08-15T17:59:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f38177fe700 1 -- 172.21.15.38:0/1607446032 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38080117c0 con 0x7f3818067030 2024-08-15T17:59:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381dd2e700 1 -- 172.21.15.38:0/1607446032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3818067030 msgr2=0x7f3818067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381dd2e700 1 --2- 172.21.15.38:0/1607446032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3818067030 0x7f3818067400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f38080098e0 tx=0x7f38080091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381dd2e700 1 -- 172.21.15.38:0/1607446032 shutdown_connections 2024-08-15T17:59:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381dd2e700 1 --2- 172.21.15.38:0/1607446032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3818067030 0x7f3818067400 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381dd2e700 1 -- 172.21.15.38:0/1607446032 >> 172.21.15.38:0/1607446032 conn(0x7f38180d0b50 msgr2=0x7f38180ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381dd2e700 1 -- 172.21.15.38:0/1607446032 shutdown_connections 2024-08-15T17:59:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.929+0000 7f381dd2e700 1 -- 172.21.15.38:0/1607446032 wait complete. 2024-08-15T17:59:17.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.933+0000 7f381dd2e700 1 Processor -- start 2024-08-15T17:59:17.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.933+0000 7f381dd2e700 1 -- start start 2024-08-15T17:59:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.933+0000 7f381dd2e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3818067030 0x7f38180b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.933+0000 7f381dd2e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38180cfb00 0x7f381814eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.933+0000 7f381dd2e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3818158cc0 0x7f381814dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.933+0000 7f381dd2e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381816d8f0 con 0x7f3818067030 2024-08-15T17:59:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.933+0000 7f381dd2e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381816cd70 con 0x7f3818158cc0 2024-08-15T17:59:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.933+0000 7f381dd2e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381816d050 con 0x7f38180cfb00 2024-08-15T17:59:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381cd2c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3818067030 0x7f38180b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381cd2c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3818067030 0x7f38180b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41636/0 (socket says 172.21.15.38:41636) 2024-08-15T17:59:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381cd2c700 1 -- 172.21.15.38:0/3454934291 learned_addr learned my addr 172.21.15.38:0/3454934291 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f3817fff700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38180cfb00 0x7f381814eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381cd2c700 1 -- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38180cfb00 msgr2=0x7f381814eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381cd2c700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38180cfb00 0x7f381814eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381cd2c700 1 -- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3818158cc0 msgr2=0x7f381814dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381cd2c700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3818158cc0 0x7f381814dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381cd2c700 1 -- 172.21.15.38:0/3454934291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3808009420 con 0x7f3818067030 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381cd2c700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3818067030 0x7f38180b4800 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3818069520 tx=0x7f3808016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f3814ff9700 1 -- 172.21.15.38:0/3454934291 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380801cbe0 con 0x7f3818067030 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f3814ff9700 1 -- 172.21.15.38:0/3454934291 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380801cd40 con 0x7f3818067030 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381dd2e700 1 -- 172.21.15.38:0/3454934291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381816d330 con 0x7f3818067030 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f3814ff9700 1 -- 172.21.15.38:0/3454934291 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380801a6a0 con 0x7f3818067030 2024-08-15T17:59:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381dd2e700 1 -- 172.21.15.38:0/3454934291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38180b51f0 con 0x7f3818067030 2024-08-15T17:59:17.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f381dd2e700 1 -- 172.21.15.38:0/3454934291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38180b4e30 con 0x7f3818067030 2024-08-15T17:59:17.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.937+0000 7f3814ff9700 1 -- 172.21.15.38:0/3454934291 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f380801aa50 con 0x7f3818067030 2024-08-15T17:59:17.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.941+0000 7f3814ff9700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f37f8060860 0x7f37f8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:17.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.941+0000 7f3814ff9700 1 -- 172.21.15.38:0/3454934291 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f380807e280 con 0x7f3818067030 2024-08-15T17:59:17.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.941+0000 7f3817fff700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f37f8060860 0x7f37f8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:17.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.941+0000 7f3817fff700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f37f8060860 0x7f37f8062d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3800001ed0 tx=0x7f3800000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:17.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:17.945+0000 7f3814ff9700 1 -- 172.21.15.38:0/3454934291 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f380804e980 con 0x7f3818067030 2024-08-15T17:59:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 -- 172.21.15.38:0/3454934291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38180638f0 con 0x7f3818067030 2024-08-15T17:59:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f3814ff9700 1 -- 172.21.15.38:0/3454934291 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3808051fa0 con 0x7f3818067030 2024-08-15T17:59:18.120 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T17:59:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 -- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f37f8060860 msgr2=0x7f37f8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f37f8060860 0x7f37f8062d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3800001ed0 tx=0x7f3800000bc0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 -- 172.21.15.38:0/3454934291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3818067030 msgr2=0x7f38180b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3818067030 0x7f38180b4800 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3818069520 tx=0x7f3808016000 comp rx=0 tx=0).stop 2024-08-15T17:59:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 -- 172.21.15.38:0/3454934291 shutdown_connections 2024-08-15T17:59:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f37f8060860 0x7f37f8062d10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3818158cc0 0x7f381814dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38180cfb00 0x7f381814eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 --2- 172.21.15.38:0/3454934291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3818067030 0x7f38180b4800 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 -- 172.21.15.38:0/3454934291 >> 172.21.15.38:0/3454934291 conn(0x7f38180d0b50 msgr2=0x7f38180b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 -- 172.21.15.38:0/3454934291 shutdown_connections 2024-08-15T17:59:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.117+0000 7f381dd2e700 1 -- 172.21.15.38:0/3454934291 wait complete. 2024-08-15T17:59:18.134 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-08-15T17:59:18.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.237+0000 7f949ec69700 1 Processor -- start 2024-08-15T17:59:18.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.237+0000 7f949ec69700 1 -- start start 2024-08-15T17:59:18.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.237+0000 7f949ec69700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f94980d0010 0x7f94980d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.237+0000 7f949ec69700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f949815cea0 0x7f949815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.237+0000 7f949ec69700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94980cf190 con 0x7f9498159040 2024-08-15T17:59:18.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.237+0000 7f949ec69700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94980d09a0 con 0x7f949815cea0 2024-08-15T17:59:18.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.237+0000 7f949ec69700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94980ce5e0 con 0x7f94980d0010 2024-08-15T17:59:18.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949dc67700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9498159040 0x7f94980cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59166/0 (socket says 172.21.15.38:59166) 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949dc67700 1 -- 172.21.15.38:0/1230824640 learned_addr learned my addr 172.21.15.38:0/1230824640 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949e468700 1 --2- 172.21.15.38:0/1230824640 >> v2:172.21.15.104:3300/0 conn(0x7f949815cea0 0x7f949815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949d466700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f94980d0010 0x7f94980d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069129605 0 0) 0x7f94980cf190 con 0x7f9498159040 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9480003570 con 0x7f9498159040 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 361712267 0 0) 0x7f9480003570 con 0x7f9498159040 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f94980d0010 msgr2=0x7f94980d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f94980d0010 0x7f94980d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 >> v2:172.21.15.104:3300/0 conn(0x7f949815cea0 msgr2=0x7f949815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 --2- 172.21.15.38:0/1230824640 >> v2:172.21.15.104:3300/0 conn(0x7f949815cea0 0x7f949815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94980ce860 con 0x7f9498159040 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f9488001470 con 0x7f9498159040 2024-08-15T17:59:18.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 >> v1:172.21.15.38:6789/0 conn(0x7f9498159040 legacy=0x7f94980cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9480003a60 0x7f9480003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9480004810 0x7f9480004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f94980d0010 0x7f94800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94980d09a0 con 0x7f9480003a60 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94980ce5e0 con 0x7f9480004810 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948000bae0 con 0x7f94980d0010 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949e468700 1 --2- 172.21.15.38:0/1230824640 >> v2:172.21.15.104:3300/0 conn(0x7f949815cea0 0x7f949815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949dc67700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9480003a60 0x7f9480003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949d466700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9480004810 0x7f9480004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949e468700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f94980d0010 0x7f94800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949dc67700 1 -- 172.21.15.38:0/1230824640 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f94980d0010 msgr2=0x7f94800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949dc67700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f94980d0010 0x7f94800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949dc67700 1 -- 172.21.15.38:0/1230824640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9480004810 msgr2=0x7f9480004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:18.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949dc67700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9480004810 0x7f9480004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949dc67700 1 -- 172.21.15.38:0/1230824640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9480003570 con 0x7f9480003a60 2024-08-15T17:59:18.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949dc67700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9480003a60 0x7f9480003e70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f948800c3c0 tx=0x7f9488001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82abf583c48b53f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:18.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948800cc90 con 0x7f9480003a60 2024-08-15T17:59:18.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948800ce30 con 0x7f9480003a60 2024-08-15T17:59:18.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949cc65700 1 -- 172.21.15.38:0/1230824640 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488005b70 con 0x7f9480003a60 2024-08-15T17:59:18.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949ec69700 1 -- 172.21.15.38:0/1230824640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9480003a60 msgr2=0x7f9480003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949ec69700 1 --2- 172.21.15.38:0/1230824640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9480003a60 0x7f9480003e70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f948800c3c0 tx=0x7f9488001e90 comp rx=0 tx=0).stop 2024-08-15T17:59:18.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949e468700 1 -- 172.21.15.38:0/1230824640 reap_dead start 2024-08-15T17:59:18.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949ec69700 1 -- 172.21.15.38:0/1230824640 shutdown_connections 2024-08-15T17:59:18.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949ec69700 1 -- 172.21.15.38:0/1230824640 >> 172.21.15.38:0/1230824640 conn(0x7f9498068250 msgr2=0x7f9498068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:18.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949ec69700 1 -- 172.21.15.38:0/1230824640 shutdown_connections 2024-08-15T17:59:18.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.241+0000 7f949ec69700 1 -- 172.21.15.38:0/1230824640 wait complete. 2024-08-15T17:59:18.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949ec69700 1 Processor -- start 2024-08-15T17:59:18.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949ec69700 1 -- start start 2024-08-15T17:59:18.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949ec69700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9480003a60 0x7f94980a7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949ec69700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94980b1980 0x7f9498147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949ec69700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498157120 0x7f9498157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949ec69700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94981480d0 con 0x7f9498157120 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949ec69700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9498157ad0 con 0x7f94980b1980 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949ec69700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94980a8660 con 0x7f9480003a60 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949e468700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498157120 0x7f9498157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949e468700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498157120 0x7f9498157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41652/0 (socket says 172.21.15.38:41652) 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949e468700 1 -- 172.21.15.38:0/2398164913 learned_addr learned my addr 172.21.15.38:0/2398164913 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949dc67700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9480003a60 0x7f94980a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949d466700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94980b1980 0x7f9498147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949e468700 1 -- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9480003a60 msgr2=0x7f94980a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949e468700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9480003a60 0x7f94980a7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949e468700 1 -- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94980b1980 msgr2=0x7f9498147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949e468700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94980b1980 0x7f9498147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949e468700 1 -- 172.21.15.38:0/2398164913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94980a7a50 con 0x7f9498157120 2024-08-15T17:59:18.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.245+0000 7f949e468700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498157120 0x7f9498157590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f94900094d0 tx=0x7f949000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:18.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.249+0000 7f9495ffb700 1 -- 172.21.15.38:0/2398164913 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949000e040 con 0x7f9498157120 2024-08-15T17:59:18.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.249+0000 7f949ec69700 1 -- 172.21.15.38:0/2398164913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94980a7d30 con 0x7f9498157120 2024-08-15T17:59:18.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.249+0000 7f9495ffb700 1 -- 172.21.15.38:0/2398164913 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949000f040 con 0x7f9498157120 2024-08-15T17:59:18.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.249+0000 7f949ec69700 1 -- 172.21.15.38:0/2398164913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94980a82f0 con 0x7f9498157120 2024-08-15T17:59:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.249+0000 7f9495ffb700 1 -- 172.21.15.38:0/2398164913 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949000fe70 con 0x7f9498157120 2024-08-15T17:59:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.249+0000 7f9495ffb700 1 -- 172.21.15.38:0/2398164913 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9490021670 con 0x7f9498157120 2024-08-15T17:59:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.249+0000 7f9495ffb700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9484060860 0x7f9484062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.249+0000 7f949ec69700 1 -- 172.21.15.38:0/2398164913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94980a87f0 con 0x7f9498157120 2024-08-15T17:59:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.249+0000 7f9495ffb700 1 -- 172.21.15.38:0/2398164913 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f949007d1c0 con 0x7f9498157120 2024-08-15T17:59:18.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.249+0000 7f949dc67700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9484060860 0x7f9484062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.249+0000 7f949dc67700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9484060860 0x7f9484062d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f948800c3c0 tx=0x7f9488015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:18.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.257+0000 7f9495ffb700 1 -- 172.21.15.38:0/2398164913 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9490077070 con 0x7f9498157120 2024-08-15T17:59:18.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.421+0000 7f949ec69700 1 -- 172.21.15.38:0/2398164913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f94980643a0 con 0x7f9498157120 2024-08-15T17:59:18.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f9495ffb700 1 -- 172.21.15.38:0/2398164913 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9490051160 con 0x7f9498157120 2024-08-15T17:59:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:18.428 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-08-15T17:59:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-08-15T17:59:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-08-15T17:59:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:18.428 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-08-15T17:59:18.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 -- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9484060860 msgr2=0x7f9484062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9484060860 0x7f9484062d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f948800c3c0 tx=0x7f9488015000 comp rx=0 tx=0).stop 2024-08-15T17:59:18.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 -- 172.21.15.38:0/2398164913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498157120 msgr2=0x7f9498157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498157120 0x7f9498157590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f94900094d0 tx=0x7f949000b040 comp rx=0 tx=0).stop 2024-08-15T17:59:18.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 -- 172.21.15.38:0/2398164913 shutdown_connections 2024-08-15T17:59:18.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9484060860 0x7f9484062d10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498157120 0x7f9498157590 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94980b1980 0x7f9498147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 --2- 172.21.15.38:0/2398164913 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9480003a60 0x7f94980a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 -- 172.21.15.38:0/2398164913 >> 172.21.15.38:0/2398164913 conn(0x7f9498068250 msgr2=0x7f9498153e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:18.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 -- 172.21.15.38:0/2398164913 shutdown_connections 2024-08-15T17:59:18.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.425+0000 7f949ec69700 1 -- 172.21.15.38:0/2398164913 wait complete. 2024-08-15T17:59:18.441 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-15T17:59:18.441 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-15T17:59:18.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.553+0000 7f56f153c700 1 Processor -- start 2024-08-15T17:59:18.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.553+0000 7f56f153c700 1 -- start start 2024-08-15T17:59:18.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f153c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f56ec155520 0x7f56ec0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f153c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec06dec0 con 0x7f56ec1593c0 2024-08-15T17:59:18.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f153c700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec0cbab0 con 0x7f56ec159f20 2024-08-15T17:59:18.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f153c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ec0ae220 con 0x7f56ec155520 2024-08-15T17:59:18.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eb7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f56ec1593c0 0x7f56ec159810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59174/0 (socket says 172.21.15.38:59174) 2024-08-15T17:59:18.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eb7fe700 1 -- 172.21.15.38:0/4012758545 learned_addr learned my addr 172.21.15.38:0/4012758545 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:18.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56ebfff700 1 --2- 172.21.15.38:0/4012758545 >> v2:172.21.15.104:3300/0 conn(0x7f56ec155520 0x7f56ec0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064678015 0 0) 0x7f56ec06dec0 con 0x7f56ec1593c0 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d8003570 con 0x7f56ec1593c0 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 314603542 0 0) 0x7f56d8003570 con 0x7f56ec1593c0 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 >> v2:172.21.15.104:3300/0 conn(0x7f56ec155520 msgr2=0x7f56ec0cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 --2- 172.21.15.38:0/4012758545 >> v2:172.21.15.104:3300/0 conn(0x7f56ec155520 0x7f56ec0cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 >> v1:172.21.15.104:6789/0 conn(0x7f56ec159f20 legacy=0x7f56ec06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec0ae500 con 0x7f56ec1593c0 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f56d4001470 con 0x7f56ec1593c0 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 >> v1:172.21.15.38:6789/0 conn(0x7f56ec1593c0 legacy=0x7f56ec159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 --2- 172.21.15.38:0/4012758545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56d8003a60 0x7f56d8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 --2- 172.21.15.38:0/4012758545 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56d8004810 0x7f56d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 --2- 172.21.15.38:0/4012758545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d80051c0 0x7f56d800dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0ae220 con 0x7f56d8003a60 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d800e370 con 0x7f56d80051c0 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d800e530 con 0x7f56d8004810 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56ebfff700 1 --2- 172.21.15.38:0/4012758545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56d8003a60 0x7f56d8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f0d3b700 1 --2- 172.21.15.38:0/4012758545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d80051c0 0x7f56d800dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eb7fe700 1 --2- 172.21.15.38:0/4012758545 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56d8004810 0x7f56d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f0d3b700 1 -- 172.21.15.38:0/4012758545 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56d8004810 msgr2=0x7f56d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f0d3b700 1 --2- 172.21.15.38:0/4012758545 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56d8004810 0x7f56d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f0d3b700 1 -- 172.21.15.38:0/4012758545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56d8003a60 msgr2=0x7f56d8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f0d3b700 1 --2- 172.21.15.38:0/4012758545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56d8003a60 0x7f56d8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f0d3b700 1 -- 172.21.15.38:0/4012758545 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d8003570 con 0x7f56d80051c0 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f0d3b700 1 -- 172.21.15.38:0/4012758545 reap_dead start 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56f0d3b700 1 --2- 172.21.15.38:0/4012758545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d80051c0 0x7f56d800dc70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f56e4001fc0 tx=0x7f56e4001640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed6a035c606113e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4001950 con 0x7f56d80051c0 2024-08-15T17:59:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e400c8b0 con 0x7f56d80051c0 2024-08-15T17:59:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.557+0000 7f56eaffd700 1 -- 172.21.15.38:0/4012758545 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e400cc50 con 0x7f56d80051c0 2024-08-15T17:59:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 -- 172.21.15.38:0/4012758545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d80051c0 msgr2=0x7f56d800dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 --2- 172.21.15.38:0/4012758545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d80051c0 0x7f56d800dc70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f56e4001fc0 tx=0x7f56e4001640 comp rx=0 tx=0).stop 2024-08-15T17:59:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 -- 172.21.15.38:0/4012758545 shutdown_connections 2024-08-15T17:59:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 --2- 172.21.15.38:0/4012758545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d80051c0 0x7f56d800dc70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 -- 172.21.15.38:0/4012758545 >> 172.21.15.38:0/4012758545 conn(0x7f56ec066e00 msgr2=0x7f56ec067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 -- 172.21.15.38:0/4012758545 shutdown_connections 2024-08-15T17:59:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 -- 172.21.15.38:0/4012758545 wait complete. 2024-08-15T17:59:18.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 Processor -- start 2024-08-15T17:59:18.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 -- start start 2024-08-15T17:59:18.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56d80051c0 0x7f56ec1529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ec152f30 0x7f56ec0ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56ec0abb30 0x7f56ec0abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec154030 con 0x7f56ec152f30 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0ac4e0 con 0x7f56ec0abb30 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.561+0000 7f56f153c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec153790 con 0x7f56d80051c0 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ec152f30 0x7f56ec0ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ec152f30 0x7f56ec0ab5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41658/0 (socket says 172.21.15.38:41658) 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56eb7fe700 1 -- 172.21.15.38:0/2616352082 learned_addr learned my addr 172.21.15.38:0/2616352082 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56ebfff700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56d80051c0 0x7f56ec1529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56f0d3b700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56ec0abb30 0x7f56ec0abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56eb7fe700 1 -- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56d80051c0 msgr2=0x7f56ec1529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56eb7fe700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56d80051c0 0x7f56ec1529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56eb7fe700 1 -- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56ec0abb30 msgr2=0x7f56ec0abfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56eb7fe700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56ec0abb30 0x7f56ec0abfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56eb7fe700 1 -- 172.21.15.38:0/2616352082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec153a70 con 0x7f56ec152f30 2024-08-15T17:59:18.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56eb7fe700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ec152f30 0x7f56ec0ab5f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f56d40073c0 tx=0x7f56d4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:18.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56e3fff700 1 -- 172.21.15.38:0/2616352082 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4018030 con 0x7f56ec152f30 2024-08-15T17:59:18.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56e3fff700 1 -- 172.21.15.38:0/2616352082 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d400f740 con 0x7f56ec152f30 2024-08-15T17:59:18.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56f153c700 1 -- 172.21.15.38:0/2616352082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec153d50 con 0x7f56ec152f30 2024-08-15T17:59:18.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56e3fff700 1 -- 172.21.15.38:0/2616352082 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d400fb00 con 0x7f56ec152f30 2024-08-15T17:59:18.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56f153c700 1 -- 172.21.15.38:0/2616352082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec13b240 con 0x7f56ec152f30 2024-08-15T17:59:18.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.565+0000 7f56f153c700 1 -- 172.21.15.38:0/2616352082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ec0644b0 con 0x7f56ec152f30 2024-08-15T17:59:18.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.569+0000 7f56e3fff700 1 -- 172.21.15.38:0/2616352082 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f56d401b070 con 0x7f56ec152f30 2024-08-15T17:59:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.569+0000 7f56e3fff700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f56cc060860 0x7f56cc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.569+0000 7f56e3fff700 1 -- 172.21.15.38:0/2616352082 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f56d4019070 con 0x7f56ec152f30 2024-08-15T17:59:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.569+0000 7f56ebfff700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f56cc060860 0x7f56cc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.569+0000 7f56ebfff700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f56cc060860 0x7f56cc062d10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56ec16cc50 tx=0x7f56dc0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:18.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.577+0000 7f56e3fff700 1 -- 172.21.15.38:0/2616352082 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f56d404cef0 con 0x7f56ec152f30 2024-08-15T17:59:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.745+0000 7f56f153c700 1 -- 172.21.15.38:0/2616352082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f56ec0af190 con 0x7f56ec152f30 2024-08-15T17:59:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.965+0000 7f56e3fff700 1 -- 172.21.15.38:0/2616352082 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f56d4050510 con 0x7f56ec152f30 2024-08-15T17:59:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 -- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f56cc060860 msgr2=0x7f56cc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f56cc060860 0x7f56cc062d10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56ec16cc50 tx=0x7f56dc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 -- 172.21.15.38:0/2616352082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ec152f30 msgr2=0x7f56ec0ab5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ec152f30 0x7f56ec0ab5f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f56d40073c0 tx=0x7f56d4001e50 comp rx=0 tx=0).stop 2024-08-15T17:59:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 -- 172.21.15.38:0/2616352082 shutdown_connections 2024-08-15T17:59:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f56cc060860 0x7f56cc062d10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56ec0abb30 0x7f56ec0abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ec152f30 0x7f56ec0ab5f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 --2- 172.21.15.38:0/2616352082 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56d80051c0 0x7f56ec1529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 -- 172.21.15.38:0/2616352082 >> 172.21.15.38:0/2616352082 conn(0x7f56ec066e00 msgr2=0x7f56ec0b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 -- 172.21.15.38:0/2616352082 shutdown_connections 2024-08-15T17:59:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:18.977+0000 7f56f153c700 1 -- 172.21.15.38:0/2616352082 wait complete. 2024-08-15T17:59:18.982 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-08-15T17:59:18.993 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-15T17:59:19.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543d5a7700 1 Processor -- start 2024-08-15T17:59:19.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543d5a7700 1 -- start start 2024-08-15T17:59:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543d5a7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f543806e8c0 0x7f54380ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543d5a7700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f54380aeb10 0x7f543813f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543d5a7700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54381573c0 con 0x7f54380adec0 2024-08-15T17:59:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543d5a7700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54380aefb0 con 0x7f54380aeb10 2024-08-15T17:59:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543d5a7700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5438156480 con 0x7f543806e8c0 2024-08-15T17:59:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543cda6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f54380aeb10 0x7f543813f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543cda6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f54380aeb10 0x7f543813f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41660/0 (socket says 172.21.15.38:41660) 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543cda6700 1 -- 172.21.15.38:0/2153549908 learned_addr learned my addr 172.21.15.38:0/2153549908 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f54377fe700 1 --2- 172.21.15.38:0/2153549908 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f543806e8c0 0x7f54380ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543cda6700 1 -- 172.21.15.38:0/2153549908 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f543806e8c0 msgr2=0x7f54380ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543cda6700 1 --2- 172.21.15.38:0/2153549908 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f543806e8c0 0x7f54380ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543cda6700 1 -- 172.21.15.38:0/2153549908 >> v1:172.21.15.104:6789/0 conn(0x7f54380adec0 legacy=0x7f543806e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543cda6700 1 -- 172.21.15.38:0/2153549908 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5438156740 con 0x7f54380aeb10 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f54377fe700 1 --2- 172.21.15.38:0/2153549908 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f543806e8c0 0x7f54380ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f543cda6700 1 --2- 172.21.15.38:0/2153549908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380aeb10 0x7f543813f1b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5430009f50 tx=0x7f5430009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b897c37504eac8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f5436ffd700 1 -- 172.21.15.38:0/2153549908 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543000e070 con 0x7f54380aeb10 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f5436ffd700 1 -- 172.21.15.38:0/2153549908 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543000ba70 con 0x7f54380aeb10 2024-08-15T17:59:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.105+0000 7f5436ffd700 1 -- 172.21.15.38:0/2153549908 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300092e0 con 0x7f54380aeb10 2024-08-15T17:59:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 -- 172.21.15.38:0/2153549908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380aeb10 msgr2=0x7f543813f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 --2- 172.21.15.38:0/2153549908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380aeb10 0x7f543813f1b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5430009f50 tx=0x7f5430009840 comp rx=0 tx=0).stop 2024-08-15T17:59:19.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 -- 172.21.15.38:0/2153549908 shutdown_connections 2024-08-15T17:59:19.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 --2- 172.21.15.38:0/2153549908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380aeb10 0x7f543813f1b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:19.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 --2- 172.21.15.38:0/2153549908 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f543806e8c0 0x7f54380ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:19.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 -- 172.21.15.38:0/2153549908 >> 172.21.15.38:0/2153549908 conn(0x7f5438067880 msgr2=0x7f5438067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:19.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 -- 172.21.15.38:0/2153549908 shutdown_connections 2024-08-15T17:59:19.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 -- 172.21.15.38:0/2153549908 wait complete. 2024-08-15T17:59:19.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 Processor -- start 2024-08-15T17:59:19.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 -- start start 2024-08-15T17:59:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f543806e8c0 0x7f54380a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54380adec0 0x7f54380a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54380aeb10 0x7f5438169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:19.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54380aa700 con 0x7f543806e8c0 2024-08-15T17:59:19.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54380aa9e0 con 0x7f54380aeb10 2024-08-15T17:59:19.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.109+0000 7f543d5a7700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54380aacc0 con 0x7f54380adec0 2024-08-15T17:59:19.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f5437fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f543806e8c0 0x7f54380a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:19.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f5437fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f543806e8c0 0x7f54380a9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41664/0 (socket says 172.21.15.38:41664) 2024-08-15T17:59:19.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f5437fff700 1 -- 172.21.15.38:0/2692840509 learned_addr learned my addr 172.21.15.38:0/2692840509 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:19.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f543cda6700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54380aeb10 0x7f5438169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:19.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f54377fe700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54380adec0 0x7f54380a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:19.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f5437fff700 1 -- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54380adec0 msgr2=0x7f54380a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:19.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f5437fff700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54380adec0 0x7f54380a9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:19.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f5437fff700 1 -- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54380aeb10 msgr2=0x7f5438169640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:19.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f5437fff700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54380aeb10 0x7f5438169640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:19.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f5437fff700 1 -- 172.21.15.38:0/2692840509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5430009ae0 con 0x7f543806e8c0 2024-08-15T17:59:19.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f5437fff700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f543806e8c0 0x7f54380a9980 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f542800c590 tx=0x7f542800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:19.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f542ffff700 1 -- 172.21.15.38:0/2692840509 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428011590 con 0x7f543806e8c0 2024-08-15T17:59:19.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f542ffff700 1 -- 172.21.15.38:0/2692840509 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5428011710 con 0x7f543806e8c0 2024-08-15T17:59:19.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f543d5a7700 1 -- 172.21.15.38:0/2692840509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54380aafa0 con 0x7f543806e8c0 2024-08-15T17:59:19.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f542ffff700 1 -- 172.21.15.38:0/2692840509 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428011ad0 con 0x7f543806e8c0 2024-08-15T17:59:19.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f543d5a7700 1 -- 172.21.15.38:0/2692840509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5438155e90 con 0x7f543806e8c0 2024-08-15T17:59:19.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f542ffff700 1 -- 172.21.15.38:0/2692840509 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f542801b3b0 con 0x7f543806e8c0 2024-08-15T17:59:19.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.113+0000 7f543d5a7700 1 -- 172.21.15.38:0/2692840509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54380643d0 con 0x7f543806e8c0 2024-08-15T17:59:19.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.117+0000 7f542ffff700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5418060860 0x7f5418062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:19.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.117+0000 7f542ffff700 1 -- 172.21.15.38:0/2692840509 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f542807bf20 con 0x7f543806e8c0 2024-08-15T17:59:19.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.117+0000 7f54377fe700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5418060860 0x7f5418062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:19.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.117+0000 7f54377fe700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5418060860 0x7f5418062d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f543806a620 tx=0x7f5420008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:19.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.121+0000 7f542ffff700 1 -- 172.21.15.38:0/2692840509 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f542804c4f0 con 0x7f543806e8c0 2024-08-15T17:59:19.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.285+0000 7f543d5a7700 1 -- 172.21.15.38:0/2692840509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f543815ad30 con 0x7f543806e8c0 2024-08-15T17:59:19.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.981+0000 7f542ffff700 1 -- 172.21.15.38:0/2692840509 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5428017020 con 0x7f543806e8c0 2024-08-15T17:59:19.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.993+0000 7f543d5a7700 1 -- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5418060860 msgr2=0x7f5418062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:19.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.993+0000 7f543d5a7700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5418060860 0x7f5418062d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f543806a620 tx=0x7f5420008040 comp rx=0 tx=0).stop 2024-08-15T17:59:19.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.993+0000 7f543d5a7700 1 -- 172.21.15.38:0/2692840509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f543806e8c0 msgr2=0x7f54380a9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:19.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.993+0000 7f543d5a7700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f543806e8c0 0x7f54380a9980 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f542800c590 tx=0x7f542800c8a0 comp rx=0 tx=0).stop 2024-08-15T17:59:19.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.997+0000 7f543d5a7700 1 -- 172.21.15.38:0/2692840509 shutdown_connections 2024-08-15T17:59:19.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.997+0000 7f543d5a7700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5418060860 0x7f5418062d10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:19.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.997+0000 7f543d5a7700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54380aeb10 0x7f5438169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:19.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.997+0000 7f543d5a7700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54380adec0 0x7f54380a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:19.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.997+0000 7f543d5a7700 1 --2- 172.21.15.38:0/2692840509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f543806e8c0 0x7f54380a9980 secure :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f542800c590 tx=0x7f542800c8a0 comp rx=0 tx=0).stop 2024-08-15T17:59:19.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.997+0000 7f543d5a7700 1 -- 172.21.15.38:0/2692840509 >> 172.21.15.38:0/2692840509 conn(0x7f5438067880 msgr2=0x7f543806cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:20.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.997+0000 7f543d5a7700 1 -- 172.21.15.38:0/2692840509 shutdown_connections 2024-08-15T17:59:20.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:19.997+0000 7f543d5a7700 1 -- 172.21.15.38:0/2692840509 wait complete. 2024-08-15T17:59:20.004 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-08-15T17:59:20.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-15T17:59:20.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.121+0000 7f8322c93700 1 Processor -- start 2024-08-15T17:59:20.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.121+0000 7f8322c93700 1 -- start start 2024-08-15T17:59:20.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f831c15a500 0x7f831c0ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:20.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f831c068db0 0x7f831c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:20.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831c06ed10 con 0x7f831c15a8d0 2024-08-15T17:59:20.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831c0b1dc0 con 0x7f831c15a500 2024-08-15T17:59:20.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831c06ea30 con 0x7f831c068db0 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8320a2f700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f831c15a500 0x7f831c0ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8320a2f700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f831c15a500 0x7f831c0ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41666/0 (socket says 172.21.15.38:41666) 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f831bfff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f831c15a8d0 0x7f831c0686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59178/0 (socket says 172.21.15.38:59178) 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8320a2f700 1 -- 172.21.15.38:0/2241672055 learned_addr learned my addr 172.21.15.38:0/2241672055 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8321230700 1 --2- 172.21.15.38:0/2241672055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f831c068db0 0x7f831c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8320a2f700 1 -- 172.21.15.38:0/2241672055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f831c068db0 msgr2=0x7f831c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8320a2f700 1 --2- 172.21.15.38:0/2241672055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f831c068db0 0x7f831c0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8320a2f700 1 -- 172.21.15.38:0/2241672055 >> v1:172.21.15.38:6789/0 conn(0x7f831c15a8d0 legacy=0x7f831c0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8320a2f700 1 -- 172.21.15.38:0/2241672055 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c0b1770 con 0x7f831c15a500 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8320a2f700 1 --2- 172.21.15.38:0/2241672055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f831c15a500 0x7f831c0ae830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f830c0099b0 tx=0x7f830c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529557fdcb33c4bb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f831b7fe700 1 -- 172.21.15.38:0/2241672055 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c00b860 con 0x7f831c15a500 2024-08-15T17:59:20.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f831b7fe700 1 -- 172.21.15.38:0/2241672055 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830c00b9c0 con 0x7f831c15a500 2024-08-15T17:59:20.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f831b7fe700 1 -- 172.21.15.38:0/2241672055 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c00bd60 con 0x7f831c15a500 2024-08-15T17:59:20.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 -- 172.21.15.38:0/2241672055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f831c15a500 msgr2=0x7f831c0ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:20.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 --2- 172.21.15.38:0/2241672055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f831c15a500 0x7f831c0ae830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f830c0099b0 tx=0x7f830c009250 comp rx=0 tx=0).stop 2024-08-15T17:59:20.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 -- 172.21.15.38:0/2241672055 shutdown_connections 2024-08-15T17:59:20.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 --2- 172.21.15.38:0/2241672055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f831c068db0 0x7f831c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:20.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 --2- 172.21.15.38:0/2241672055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f831c15a500 0x7f831c0ae830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:20.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 -- 172.21.15.38:0/2241672055 >> 172.21.15.38:0/2241672055 conn(0x7f831c0c9bf0 msgr2=0x7f831c06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:20.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 -- 172.21.15.38:0/2241672055 shutdown_connections 2024-08-15T17:59:20.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.125+0000 7f8322c93700 1 -- 172.21.15.38:0/2241672055 wait complete. 2024-08-15T17:59:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8322c93700 1 Processor -- start 2024-08-15T17:59:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8322c93700 1 -- start start 2024-08-15T17:59:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8322c93700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f831c068db0 0x7f831c0a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8322c93700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f831c15a500 0x7f831c067d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8322c93700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f831c15a8d0 0x7f831c068270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8322c93700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c1684e0 con 0x7f831c15a500 2024-08-15T17:59:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8322c93700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c16aca0 con 0x7f831c068db0 2024-08-15T17:59:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8322c93700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831c16af80 con 0x7f831c15a8d0 2024-08-15T17:59:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f831bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f831c15a500 0x7f831c067d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f831bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f831c15a500 0x7f831c067d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41672/0 (socket says 172.21.15.38:41672) 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f831bfff700 1 -- 172.21.15.38:0/3855598028 learned_addr learned my addr 172.21.15.38:0/3855598028 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8320a2f700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f831c068db0 0x7f831c0a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8321230700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f831c15a8d0 0x7f831c068270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f831bfff700 1 -- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f831c15a8d0 msgr2=0x7f831c068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f831bfff700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f831c15a8d0 0x7f831c068270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f831bfff700 1 -- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f831c068db0 msgr2=0x7f831c0a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f831bfff700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f831c068db0 0x7f831c0a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f831bfff700 1 -- 172.21.15.38:0/3855598028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c0094f0 con 0x7f831c15a500 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8321230700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f831c15a8d0 0x7f831c068270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f831bfff700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f831c15a500 0x7f831c067d30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f831000c590 tx=0x7f8310007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8318ff9700 1 -- 172.21.15.38:0/3855598028 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310005a70 con 0x7f831c15a500 2024-08-15T17:59:20.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8318ff9700 1 -- 172.21.15.38:0/3855598028 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8310005bd0 con 0x7f831c15a500 2024-08-15T17:59:20.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8322c93700 1 -- 172.21.15.38:0/3855598028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831c16b260 con 0x7f831c15a500 2024-08-15T17:59:20.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8318ff9700 1 -- 172.21.15.38:0/3855598028 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83100070d0 con 0x7f831c15a500 2024-08-15T17:59:20.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.129+0000 7f8322c93700 1 -- 172.21.15.38:0/3855598028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831c0a89f0 con 0x7f831c15a500 2024-08-15T17:59:20.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.133+0000 7f8322c93700 1 -- 172.21.15.38:0/3855598028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831c0645c0 con 0x7f831c15a500 2024-08-15T17:59:20.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.133+0000 7f8318ff9700 1 -- 172.21.15.38:0/3855598028 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8310007440 con 0x7f831c15a500 2024-08-15T17:59:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.133+0000 7f8318ff9700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8308060860 0x7f8308062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.133+0000 7f8318ff9700 1 -- 172.21.15.38:0/3855598028 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f831007be90 con 0x7f831c15a500 2024-08-15T17:59:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.133+0000 7f8320a2f700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8308060860 0x7f8308062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.133+0000 7f8320a2f700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8308060860 0x7f8308062d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f830c0099b0 tx=0x7f830c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:20.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.141+0000 7f8318ff9700 1 -- 172.21.15.38:0/3855598028 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f831004c340 con 0x7f831c15a500 2024-08-15T17:59:20.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:20.345+0000 7f8322c93700 1 -- 172.21.15.38:0/3855598028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f831c067810 con 0x7f831c15a500 2024-08-15T17:59:21.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.001+0000 7f8318ff9700 1 -- 172.21.15.38:0/3855598028 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f831004f960 con 0x7f831c15a500 2024-08-15T17:59:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 -- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8308060860 msgr2=0x7f8308062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8308060860 0x7f8308062d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f830c0099b0 tx=0x7f830c017000 comp rx=0 tx=0).stop 2024-08-15T17:59:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 -- 172.21.15.38:0/3855598028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f831c15a500 msgr2=0x7f831c067d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f831c15a500 0x7f831c067d30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f831000c590 tx=0x7f8310007f90 comp rx=0 tx=0).stop 2024-08-15T17:59:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 -- 172.21.15.38:0/3855598028 shutdown_connections 2024-08-15T17:59:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8308060860 0x7f8308062d10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f831c15a8d0 0x7f831c068270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f831c15a500 0x7f831c067d30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 --2- 172.21.15.38:0/3855598028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f831c068db0 0x7f831c0a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 -- 172.21.15.38:0/3855598028 >> 172.21.15.38:0/3855598028 conn(0x7f831c0c9bf0 msgr2=0x7f831c0cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:21.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 -- 172.21.15.38:0/3855598028 shutdown_connections 2024-08-15T17:59:21.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.017+0000 7f8322c93700 1 -- 172.21.15.38:0/3855598028 wait complete. 2024-08-15T17:59:21.021 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 4 and data pool 5 2024-08-15T17:59:21.031 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:59:21.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74f4e75700 1 Processor -- start 2024-08-15T17:59:21.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74f4e75700 1 -- start start 2024-08-15T17:59:21.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74f4e75700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f015d540 0x7f74f00af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74f4e75700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f00b0270 con 0x7f74f006e830 2024-08-15T17:59:21.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74f4e75700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f00af660 con 0x7f74f00a9430 2024-08-15T17:59:21.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74f4e75700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00af940 con 0x7f74f015d540 2024-08-15T17:59:21.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74edd9b700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f74f006e830 0x7f74f00a8d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59194/0 (socket says 172.21.15.38:59194) 2024-08-15T17:59:21.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74edd9b700 1 -- 172.21.15.38:0/3678528929 learned_addr learned my addr 172.21.15.38:0/3678528929 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:21.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ee59c700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f015d540 0x7f74f00af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784300227 0 0) 0x7f74f00b0270 con 0x7f74f006e830 2024-08-15T17:59:21.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74d8003570 con 0x7f74f006e830 2024-08-15T17:59:21.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444673648 0 0) 0x7f74f00af660 con 0x7f74f00a9430 2024-08-15T17:59:21.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74f00b0270 con 0x7f74f00a9430 2024-08-15T17:59:21.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2607615971 0 0) 0x7f74d8003570 con 0x7f74f006e830 2024-08-15T17:59:21.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f015d540 msgr2=0x7f74f00af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:21.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f015d540 0x7f74f00af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 >> v1:172.21.15.104:6789/0 conn(0x7f74f00a9430 legacy=0x7f74f00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f00afc20 con 0x7f74f006e830 2024-08-15T17:59:21.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f74e00017f0 con 0x7f74f006e830 2024-08-15T17:59:21.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 >> v1:172.21.15.38:6789/0 conn(0x7f74f006e830 legacy=0x7f74f00a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74d8003c20 0x7f74d8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f015d540 0x7f74d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 0x7f74d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00af940 con 0x7f74d8004d50 2024-08-15T17:59:21.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d800bb40 con 0x7f74d8003c20 2024-08-15T17:59:21.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d800bd00 con 0x7f74f015d540 2024-08-15T17:59:21.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74eed9d700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 0x7f74d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74ee59c700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74d8003c20 0x7f74d8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74edd9b700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f015d540 0x7f74d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74eed9d700 1 -- 172.21.15.38:0/3678528929 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f015d540 msgr2=0x7f74d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74eed9d700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f015d540 0x7f74d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74eed9d700 1 -- 172.21.15.38:0/3678528929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74d8003c20 msgr2=0x7f74d8004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74eed9d700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74d8003c20 0x7f74d8004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74eed9d700 1 -- 172.21.15.38:0/3678528929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d8003570 con 0x7f74d8004d50 2024-08-15T17:59:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.133+0000 7f74eed9d700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 0x7f74d80051c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f74e8001a10 tx=0x7f74e8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164a8d4817d90754 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.137+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e800c5d0 con 0x7f74d8004d50 2024-08-15T17:59:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.137+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e800c730 con 0x7f74d8004d50 2024-08-15T17:59:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.137+0000 7f74ed59a700 1 -- 172.21.15.38:0/3678528929 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e800cad0 con 0x7f74d8004d50 2024-08-15T17:59:21.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.137+0000 7f74f4e75700 1 -- 172.21.15.38:0/3678528929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 msgr2=0x7f74d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.137+0000 7f74f4e75700 1 --2- 172.21.15.38:0/3678528929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 0x7f74d80051c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f74e8001a10 tx=0x7f74e8007d30 comp rx=0 tx=0).stop 2024-08-15T17:59:21.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.137+0000 7f74eed9d700 1 -- 172.21.15.38:0/3678528929 reap_dead start 2024-08-15T17:59:21.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.137+0000 7f74f4e75700 1 -- 172.21.15.38:0/3678528929 shutdown_connections 2024-08-15T17:59:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.137+0000 7f74f4e75700 1 -- 172.21.15.38:0/3678528929 >> 172.21.15.38:0/3678528929 conn(0x7f74f0066e00 msgr2=0x7f74f0067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.137+0000 7f74f4e75700 1 -- 172.21.15.38:0/3678528929 shutdown_connections 2024-08-15T17:59:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.137+0000 7f74f4e75700 1 -- 172.21.15.38:0/3678528929 wait complete. 2024-08-15T17:59:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74f4e75700 1 Processor -- start 2024-08-15T17:59:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74f4e75700 1 -- start start 2024-08-15T17:59:21.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74f4e75700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 0x7f74f015f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74f4e75700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f016df30 0x7f74f01587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74f4e75700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f006a600 0x7f74f006aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74f4e75700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f0160550 con 0x7f74d8004d50 2024-08-15T17:59:21.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74f4e75700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f006af70 con 0x7f74f006a600 2024-08-15T17:59:21.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74f4e75700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f0158d00 con 0x7f74f016df30 2024-08-15T17:59:21.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74ee59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 0x7f74f015f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74ee59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 0x7f74f015f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41680/0 (socket says 172.21.15.38:41680) 2024-08-15T17:59:21.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74ee59c700 1 -- 172.21.15.38:0/1097601124 learned_addr learned my addr 172.21.15.38:0/1097601124 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:21.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74eed9d700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f006a600 0x7f74f006aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74edd9b700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f016df30 0x7f74f01587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74ee59c700 1 -- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f016df30 msgr2=0x7f74f01587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74ee59c700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f016df30 0x7f74f01587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74ee59c700 1 -- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f006a600 msgr2=0x7f74f006aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74ee59c700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f006a600 0x7f74f006aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74ee59c700 1 -- 172.21.15.38:0/1097601124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f015f940 con 0x7f74d8004d50 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74eed9d700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f006a600 0x7f74f006aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74edd9b700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f016df30 0x7f74f01587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74ee59c700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 0x7f74f015f2f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f74e4011b10 tx=0x7f74e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74de7fc700 1 -- 172.21.15.38:0/1097601124 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e400f9f0 con 0x7f74d8004d50 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74de7fc700 1 -- 172.21.15.38:0/1097601124 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e400fb50 con 0x7f74d8004d50 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74f4e75700 1 -- 172.21.15.38:0/1097601124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f015fc20 con 0x7f74d8004d50 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74de7fc700 1 -- 172.21.15.38:0/1097601124 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e400fed0 con 0x7f74d8004d50 2024-08-15T17:59:21.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.141+0000 7f74f4e75700 1 -- 172.21.15.38:0/1097601124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f01601e0 con 0x7f74d8004d50 2024-08-15T17:59:21.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.145+0000 7f74f4e75700 1 -- 172.21.15.38:0/1097601124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f00645c0 con 0x7f74d8004d50 2024-08-15T17:59:21.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.145+0000 7f74de7fc700 1 -- 172.21.15.38:0/1097601124 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f74e4014650 con 0x7f74d8004d50 2024-08-15T17:59:21.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.145+0000 7f74de7fc700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f74d0060860 0x7f74d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.145+0000 7f74de7fc700 1 -- 172.21.15.38:0/1097601124 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f74e4018030 con 0x7f74d8004d50 2024-08-15T17:59:21.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.145+0000 7f74edd9b700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f74d0060860 0x7f74d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.145+0000 7f74edd9b700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f74d0060860 0x7f74d0062d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f74e0002000 tx=0x7f74e0006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:21.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.157+0000 7f74de7fc700 1 -- 172.21.15.38:0/1097601124 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f74e404da50 con 0x7f74d8004d50 2024-08-15T17:59:21.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.321+0000 7f74f4e75700 1 -- 172.21.15.38:0/1097601124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74f006f360 con 0x7f74d8004d50 2024-08-15T17:59:21.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.321+0000 7f74de7fc700 1 -- 172.21.15.38:0/1097601124 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f74e4051070 con 0x7f74d8004d50 2024-08-15T17:59:21.325 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:21.326 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":31,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:59:20.352672+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:59:18.793510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:59:19.328139+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:59:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 -- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f74d0060860 msgr2=0x7f74d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f74d0060860 0x7f74d0062d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f74e0002000 tx=0x7f74e0006040 comp rx=0 tx=0).stop 2024-08-15T17:59:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 -- 172.21.15.38:0/1097601124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 msgr2=0x7f74f015f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 0x7f74f015f2f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f74e4011b10 tx=0x7f74e40091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 -- 172.21.15.38:0/1097601124 shutdown_connections 2024-08-15T17:59:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f74d0060860 0x7f74d0062d10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f006a600 0x7f74f006aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74f016df30 0x7f74f01587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 --2- 172.21.15.38:0/1097601124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74d8004d50 0x7f74f015f2f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 -- 172.21.15.38:0/1097601124 >> 172.21.15.38:0/1097601124 conn(0x7f74f0066e00 msgr2=0x7f74f00671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 -- 172.21.15.38:0/1097601124 shutdown_connections 2024-08-15T17:59:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.325+0000 7f74f4e75700 1 -- 172.21.15.38:0/1097601124 wait complete. 2024-08-15T17:59:21.340 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:59:21.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.437+0000 7f84269ea700 1 Processor -- start 2024-08-15T17:59:21.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.437+0000 7f84269ea700 1 -- start start 2024-08-15T17:59:21.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.437+0000 7f84269ea700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8420155d20 0x7f84200b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.437+0000 7f84269ea700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f842006df10 0x7f842006e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.437+0000 7f84269ea700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842006e8b0 con 0x7f84200ad7c0 2024-08-15T17:59:21.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.437+0000 7f84269ea700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84200adc10 con 0x7f8420155d20 2024-08-15T17:59:21.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.437+0000 7f84269ea700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84200af2b0 con 0x7f842006df10 2024-08-15T17:59:21.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.437+0000 7f84259e8700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8420155d20 0x7f84200b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.437+0000 7f84261e9700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f842006df10 0x7f842006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.437+0000 7f84259e8700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8420155d20 0x7f84200b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:55748/0 (socket says 172.21.15.38:55748) 2024-08-15T17:59:21.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.437+0000 7f84259e8700 1 -- 172.21.15.38:0/1030141766 learned_addr learned my addr 172.21.15.38:0/1030141766 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:21.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84259e8700 1 -- 172.21.15.38:0/1030141766 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f842006df10 msgr2=0x7f842006e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:21.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84259e8700 1 --2- 172.21.15.38:0/1030141766 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f842006df10 0x7f842006e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84259e8700 1 -- 172.21.15.38:0/1030141766 >> v1:172.21.15.104:6789/0 conn(0x7f84200ad7c0 legacy=0x7f84200aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84259e8700 1 -- 172.21.15.38:0/1030141766 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200af590 con 0x7f8420155d20 2024-08-15T17:59:21.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84261e9700 1 --2- 172.21.15.38:0/1030141766 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f842006df10 0x7f842006e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:21.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84259e8700 1 --2- 172.21.15.38:0/1030141766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420155d20 0x7f84200b0350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8418009f20 tx=0x7f8418009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a72edf4553b832 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:21.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84249e6700 1 -- 172.21.15.38:0/1030141766 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800e070 con 0x7f8420155d20 2024-08-15T17:59:21.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84249e6700 1 -- 172.21.15.38:0/1030141766 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841800b9d0 con 0x7f8420155d20 2024-08-15T17:59:21.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84249e6700 1 -- 172.21.15.38:0/1030141766 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84180092e0 con 0x7f8420155d20 2024-08-15T17:59:21.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84269ea700 1 -- 172.21.15.38:0/1030141766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420155d20 msgr2=0x7f84200b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84269ea700 1 --2- 172.21.15.38:0/1030141766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420155d20 0x7f84200b0350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8418009f20 tx=0x7f8418009840 comp rx=0 tx=0).stop 2024-08-15T17:59:21.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84269ea700 1 -- 172.21.15.38:0/1030141766 shutdown_connections 2024-08-15T17:59:21.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84269ea700 1 --2- 172.21.15.38:0/1030141766 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f842006df10 0x7f842006e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84269ea700 1 --2- 172.21.15.38:0/1030141766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8420155d20 0x7f84200b0350 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84269ea700 1 -- 172.21.15.38:0/1030141766 >> 172.21.15.38:0/1030141766 conn(0x7f84200678d0 msgr2=0x7f8420067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:21.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84269ea700 1 -- 172.21.15.38:0/1030141766 shutdown_connections 2024-08-15T17:59:21.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.441+0000 7f84269ea700 1 -- 172.21.15.38:0/1030141766 wait complete. 2024-08-15T17:59:21.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84269ea700 1 Processor -- start 2024-08-15T17:59:21.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84269ea700 1 -- start start 2024-08-15T17:59:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84269ea700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f842006df10 0x7f842014f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84269ea700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84200ad7c0 0x7f8420145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84269ea700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8420155d20 0x7f84200ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84269ea700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842014fc70 con 0x7f842006df10 2024-08-15T17:59:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84269ea700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8420069560 con 0x7f84200ad7c0 2024-08-15T17:59:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84269ea700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8420168890 con 0x7f8420155d20 2024-08-15T17:59:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84259e8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f842006df10 0x7f842014f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84261e9700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8420155d20 0x7f84200ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84259e8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f842006df10 0x7f842014f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41694/0 (socket says 172.21.15.38:41694) 2024-08-15T17:59:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84259e8700 1 -- 172.21.15.38:0/3588253391 learned_addr learned my addr 172.21.15.38:0/3588253391 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.445+0000 7f84251e7700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84200ad7c0 0x7f8420145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f84259e8700 1 -- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8420155d20 msgr2=0x7f84200ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f84259e8700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8420155d20 0x7f84200ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f84259e8700 1 -- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84200ad7c0 msgr2=0x7f8420145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f84259e8700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84200ad7c0 0x7f8420145830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f84259e8700 1 -- 172.21.15.38:0/3588253391 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8410009420 con 0x7f842006df10 2024-08-15T17:59:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f84259e8700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f842006df10 0x7f842014f730 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f841800b610 tx=0x7f8418018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f841dffb700 1 -- 172.21.15.38:0/3588253391 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84180089a0 con 0x7f842006df10 2024-08-15T17:59:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f841dffb700 1 -- 172.21.15.38:0/3588253391 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8418008b00 con 0x7f842006df10 2024-08-15T17:59:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f841dffb700 1 -- 172.21.15.38:0/3588253391 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841801c410 con 0x7f842006df10 2024-08-15T17:59:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f84269ea700 1 -- 172.21.15.38:0/3588253391 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8418009ae0 con 0x7f842006df10 2024-08-15T17:59:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f84269ea700 1 -- 172.21.15.38:0/3588253391 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8420069ae0 con 0x7f842006df10 2024-08-15T17:59:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f84269ea700 1 -- 172.21.15.38:0/3588253391 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8420069d00 con 0x7f842006df10 2024-08-15T17:59:21.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f841dffb700 1 -- 172.21.15.38:0/3588253391 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8418003040 con 0x7f842006df10 2024-08-15T17:59:21.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f841dffb700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f840c060860 0x7f840c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f841dffb700 1 -- 172.21.15.38:0/3588253391 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f841807e140 con 0x7f842006df10 2024-08-15T17:59:21.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.449+0000 7f84251e7700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f840c060860 0x7f840c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.453+0000 7f84251e7700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f840c060860 0x7f840c062d10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8420069450 tx=0x7f8414007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:21.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.461+0000 7f841dffb700 1 -- 172.21.15.38:0/3588253391 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f841804e650 con 0x7f842006df10 2024-08-15T17:59:21.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.625+0000 7f84269ea700 1 -- 172.21.15.38:0/3588253391 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84200643a0 con 0x7f842006df10 2024-08-15T17:59:21.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f841dffb700 1 -- 172.21.15.38:0/3588253391 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f8418051c70 con 0x7f842006df10 2024-08-15T17:59:21.632 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:21.632 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":31,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:59:20.352672+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:59:18.793510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:59:19.328139+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:59:21.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 -- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f840c060860 msgr2=0x7f840c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f840c060860 0x7f840c062d10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8420069450 tx=0x7f8414007ec0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 -- 172.21.15.38:0/3588253391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f842006df10 msgr2=0x7f842014f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f842006df10 0x7f842014f730 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f841800b610 tx=0x7f8418018000 comp rx=0 tx=0).stop 2024-08-15T17:59:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 -- 172.21.15.38:0/3588253391 shutdown_connections 2024-08-15T17:59:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f840c060860 0x7f840c062d10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8420155d20 0x7f84200ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84200ad7c0 0x7f8420145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 --2- 172.21.15.38:0/3588253391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f842006df10 0x7f842014f730 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 -- 172.21.15.38:0/3588253391 >> 172.21.15.38:0/3588253391 conn(0x7f84200678d0 msgr2=0x7f8420153950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 -- 172.21.15.38:0/3588253391 shutdown_connections 2024-08-15T17:59:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.629+0000 7f84269ea700 1 -- 172.21.15.38:0/3588253391 wait complete. 2024-08-15T17:59:21.647 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-08-15T17:59:21.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.757+0000 7fb09e9ec700 1 Processor -- start 2024-08-15T17:59:21.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.757+0000 7fb09e9ec700 1 -- start start 2024-08-15T17:59:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb098067030 0x7fb098067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0980cfb00 0x7fb0980cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0980cf670 con 0x7fb0980cb5c0 2024-08-15T17:59:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0980cea60 con 0x7fb098067030 2024-08-15T17:59:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0980ced40 con 0x7fb0980cfb00 2024-08-15T17:59:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09d9ea700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb098067030 0x7fb098067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e1eb700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb0980cb5c0 0x7fb098158cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59204/0 (socket says 172.21.15.38:59204) 2024-08-15T17:59:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09d9ea700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb098067030 0x7fb098067400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41706/0 (socket says 172.21.15.38:41706) 2024-08-15T17:59:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e1eb700 1 -- 172.21.15.38:0/1666011020 learned_addr learned my addr 172.21.15.38:0/1666011020 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09d1e9700 1 --2- 172.21.15.38:0/1666011020 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0980cfb00 0x7fb0980cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09c9e8700 1 -- 172.21.15.38:0/1666011020 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951769162 0 0) 0x7fb0980cf670 con 0x7fb0980cb5c0 2024-08-15T17:59:21.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09d9ea700 1 -- 172.21.15.38:0/1666011020 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0980cfb00 msgr2=0x7fb0980cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:21.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09d9ea700 1 --2- 172.21.15.38:0/1666011020 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0980cfb00 0x7fb0980cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09d9ea700 1 -- 172.21.15.38:0/1666011020 >> v1:172.21.15.38:6789/0 conn(0x7fb0980cb5c0 legacy=0x7fb098158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09d9ea700 1 -- 172.21.15.38:0/1666011020 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0980cf020 con 0x7fb098067030 2024-08-15T17:59:21.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09d9ea700 1 --2- 172.21.15.38:0/1666011020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb098067030 0x7fb098067400 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb0940098e0 tx=0x7fb0940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8becf09eb1a10463 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:21.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09c9e8700 1 -- 172.21.15.38:0/1666011020 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09400b640 con 0x7fb098067030 2024-08-15T17:59:21.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09c9e8700 1 -- 172.21.15.38:0/1666011020 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09400b7a0 con 0x7fb098067030 2024-08-15T17:59:21.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09c9e8700 1 -- 172.21.15.38:0/1666011020 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09400bb40 con 0x7fb098067030 2024-08-15T17:59:21.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1666011020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb098067030 msgr2=0x7fb098067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 --2- 172.21.15.38:0/1666011020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb098067030 0x7fb098067400 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb0940098e0 tx=0x7fb0940091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1666011020 shutdown_connections 2024-08-15T17:59:21.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 --2- 172.21.15.38:0/1666011020 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0980cfb00 0x7fb0980cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 --2- 172.21.15.38:0/1666011020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb098067030 0x7fb098067400 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1666011020 >> 172.21.15.38:0/1666011020 conn(0x7fb0980d0b50 msgr2=0x7fb0980ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:21.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1666011020 shutdown_connections 2024-08-15T17:59:21.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.761+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1666011020 wait complete. 2024-08-15T17:59:21.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.765+0000 7fb09e9ec700 1 Processor -- start 2024-08-15T17:59:21.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.765+0000 7fb09e9ec700 1 -- start start 2024-08-15T17:59:21.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.765+0000 7fb09e9ec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb098067030 0x7fb098150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.765+0000 7fb09e9ec700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0980cb5c0 0x7fb09815c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.765+0000 7fb09e9ec700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0980cfb00 0x7fb0980c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.765+0000 7fb09e9ec700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098150850 con 0x7fb098067030 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.765+0000 7fb09e9ec700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09815b2b0 con 0x7fb0980cb5c0 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.765+0000 7fb09e9ec700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09815b590 con 0x7fb0980cfb00 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.765+0000 7fb09d9ea700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb098067030 0x7fb098150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09d9ea700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb098067030 0x7fb098150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41712/0 (socket says 172.21.15.38:41712) 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09d9ea700 1 -- 172.21.15.38:0/1652594623 learned_addr learned my addr 172.21.15.38:0/1652594623 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09d1e9700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0980cb5c0 0x7fb09815c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09e1eb700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0980cfb00 0x7fb0980c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09d9ea700 1 -- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0980cfb00 msgr2=0x7fb0980c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09d9ea700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0980cfb00 0x7fb0980c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09d9ea700 1 -- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0980cb5c0 msgr2=0x7fb09815c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09d9ea700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0980cb5c0 0x7fb09815c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09d9ea700 1 -- 172.21.15.38:0/1652594623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb094009420 con 0x7fb098067030 2024-08-15T17:59:21.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09d9ea700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb098067030 0x7fb098150310 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb098069520 tx=0x7fb094017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:21.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb08dffb700 1 -- 172.21.15.38:0/1652594623 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09400bb20 con 0x7fb098067030 2024-08-15T17:59:21.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb08dffb700 1 -- 172.21.15.38:0/1652594623 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb094000c90 con 0x7fb098067030 2024-08-15T17:59:21.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1652594623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09815b870 con 0x7fb098067030 2024-08-15T17:59:21.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb08dffb700 1 -- 172.21.15.38:0/1652594623 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb094011650 con 0x7fb098067030 2024-08-15T17:59:21.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1652594623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09815be30 con 0x7fb098067030 2024-08-15T17:59:21.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.769+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1652594623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09815c050 con 0x7fb098067030 2024-08-15T17:59:21.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.773+0000 7fb08dffb700 1 -- 172.21.15.38:0/1652594623 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb094011a00 con 0x7fb098067030 2024-08-15T17:59:21.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.773+0000 7fb08dffb700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb07c060860 0x7fb07c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:21.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.773+0000 7fb08dffb700 1 -- 172.21.15.38:0/1652594623 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb09407e1c0 con 0x7fb098067030 2024-08-15T17:59:21.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.773+0000 7fb09d1e9700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb07c060860 0x7fb07c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:21.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.773+0000 7fb09d1e9700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb07c060860 0x7fb07c062d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb0980cad90 tx=0x7fb0880091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:21.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.777+0000 7fb08dffb700 1 -- 172.21.15.38:0/1652594623 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb09404e610 con 0x7fb098067030 2024-08-15T17:59:21.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:21.981+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1652594623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb0980638f0 con 0x7fb098067030 2024-08-15T17:59:22.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.889+0000 7fb08dffb700 1 -- 172.21.15.38:0/1652594623 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb094011cb0 con 0x7fb098067030 2024-08-15T17:59:22.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb07c060860 msgr2=0x7fb07c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb07c060860 0x7fb07c062d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb0980cad90 tx=0x7fb0880091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1652594623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb098067030 msgr2=0x7fb098150310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb098067030 0x7fb098150310 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb098069520 tx=0x7fb094017000 comp rx=0 tx=0).stop 2024-08-15T17:59:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1652594623 shutdown_connections 2024-08-15T17:59:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb07c060860 0x7fb07c062d10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0980cfb00 0x7fb0980c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0980cb5c0 0x7fb09815c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 --2- 172.21.15.38:0/1652594623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb098067030 0x7fb098150310 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1652594623 >> 172.21.15.38:0/1652594623 conn(0x7fb0980d0b50 msgr2=0x7fb0980c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1652594623 shutdown_connections 2024-08-15T17:59:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:22.897+0000 7fb09e9ec700 1 -- 172.21.15.38:0/1652594623 wait complete. 2024-08-15T17:59:22.918 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-08-15T17:59:23.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 Processor -- start 2024-08-15T17:59:23.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 -- start start 2024-08-15T17:59:23.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa5801538e0 0x7fa5800b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:23.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa580153cb0 0x7fa58006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:23.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa58006f180 0x7fa580159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:23.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa58006f5e0 con 0x7fa580153cb0 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5800b0ad0 con 0x7fa5801538e0 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58015a940 con 0x7fa58006f180 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57ffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa580153cb0 0x7fa58006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57ffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa580153cb0 0x7fa58006ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41722/0 (socket says 172.21.15.38:41722) 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57ffff700 1 -- 172.21.15.38:0/1524623690 learned_addr learned my addr 172.21.15.38:0/1524623690 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa585062700 1 --2- 172.21.15.38:0/1524623690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa58006f180 0x7fa580159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa584861700 1 --2- 172.21.15.38:0/1524623690 >> v2:172.21.15.104:3300/0 conn(0x7fa5801538e0 0x7fa5800b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57ffff700 1 -- 172.21.15.38:0/1524623690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa58006f180 msgr2=0x7fa580159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57ffff700 1 --2- 172.21.15.38:0/1524623690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa58006f180 0x7fa580159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57ffff700 1 -- 172.21.15.38:0/1524623690 >> v2:172.21.15.104:3300/0 conn(0x7fa5801538e0 msgr2=0x7fa5800b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57ffff700 1 --2- 172.21.15.38:0/1524623690 >> v2:172.21.15.104:3300/0 conn(0x7fa5801538e0 0x7fa5800b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57ffff700 1 -- 172.21.15.38:0/1524623690 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5800b01a0 con 0x7fa580153cb0 2024-08-15T17:59:23.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa585062700 1 --2- 172.21.15.38:0/1524623690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa58006f180 0x7fa580159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:23.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57ffff700 1 --2- 172.21.15.38:0/1524623690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa580153cb0 0x7fa58006ec40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa580071080 tx=0x7fa57000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cf79c9994d016c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:23.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57f7fe700 1 -- 172.21.15.38:0/1524623690 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57000dc50 con 0x7fa580153cb0 2024-08-15T17:59:23.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57f7fe700 1 -- 172.21.15.38:0/1524623690 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57000ddb0 con 0x7fa580153cb0 2024-08-15T17:59:23.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa57f7fe700 1 -- 172.21.15.38:0/1524623690 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5700196d0 con 0x7fa580153cb0 2024-08-15T17:59:23.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 -- 172.21.15.38:0/1524623690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa580153cb0 msgr2=0x7fa58006ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:23.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 --2- 172.21.15.38:0/1524623690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa580153cb0 0x7fa58006ec40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa580071080 tx=0x7fa57000b4f0 comp rx=0 tx=0).stop 2024-08-15T17:59:23.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 -- 172.21.15.38:0/1524623690 shutdown_connections 2024-08-15T17:59:23.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 --2- 172.21.15.38:0/1524623690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa58006f180 0x7fa580159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:23.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 --2- 172.21.15.38:0/1524623690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa580153cb0 0x7fa58006ec40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:23.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 --2- 172.21.15.38:0/1524623690 >> v2:172.21.15.104:3300/0 conn(0x7fa5801538e0 0x7fa5800b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:23.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 -- 172.21.15.38:0/1524623690 >> 172.21.15.38:0/1524623690 conn(0x7fa580066e00 msgr2=0x7fa580067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:23.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.041+0000 7fa586ac5700 1 -- 172.21.15.38:0/1524623690 shutdown_connections 2024-08-15T17:59:23.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.045+0000 7fa586ac5700 1 -- 172.21.15.38:0/1524623690 wait complete. 2024-08-15T17:59:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.045+0000 7fa586ac5700 1 Processor -- start 2024-08-15T17:59:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.045+0000 7fa586ac5700 1 -- start start 2024-08-15T17:59:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa586ac5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58006f180 0x7fa5801427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa586ac5700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5801538e0 0x7fa5801526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa586ac5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa580153cb0 0x7fa580152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa586ac5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa580158230 con 0x7fa58006f180 2024-08-15T17:59:23.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa586ac5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa580157660 con 0x7fa580153cb0 2024-08-15T17:59:23.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa586ac5700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa580157900 con 0x7fa5801538e0 2024-08-15T17:59:23.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa584861700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58006f180 0x7fa5801427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:23.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa584861700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58006f180 0x7fa5801427d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41724/0 (socket says 172.21.15.38:41724) 2024-08-15T17:59:23.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa584861700 1 -- 172.21.15.38:0/2570477618 learned_addr learned my addr 172.21.15.38:0/2570477618 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:23.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa585062700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa580153cb0 0x7fa580152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:23.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa57ffff700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5801538e0 0x7fa5801526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa585062700 1 -- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5801538e0 msgr2=0x7fa5801526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa585062700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5801538e0 0x7fa5801526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa585062700 1 -- 172.21.15.38:0/2570477618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58006f180 msgr2=0x7fa5801427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa585062700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58006f180 0x7fa5801427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa585062700 1 -- 172.21.15.38:0/2570477618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57000b790 con 0x7fa580153cb0 2024-08-15T17:59:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa584861700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58006f180 0x7fa5801427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa585062700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa580153cb0 0x7fa580152c00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa578009fa0 tx=0x7fa578009990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa57cff9700 1 -- 172.21.15.38:0/2570477618 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57800dbb0 con 0x7fa580153cb0 2024-08-15T17:59:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa57cff9700 1 -- 172.21.15.38:0/2570477618 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57800f040 con 0x7fa580153cb0 2024-08-15T17:59:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa57cff9700 1 -- 172.21.15.38:0/2570477618 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa578015440 con 0x7fa580153cb0 2024-08-15T17:59:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa586ac5700 1 -- 172.21.15.38:0/2570477618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa580157be0 con 0x7fa580153cb0 2024-08-15T17:59:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.049+0000 7fa586ac5700 1 -- 172.21.15.38:0/2570477618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa580142d10 con 0x7fa580153cb0 2024-08-15T17:59:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.053+0000 7fa586ac5700 1 -- 172.21.15.38:0/2570477618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5801583a0 con 0x7fa580153cb0 2024-08-15T17:59:23.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.053+0000 7fa57cff9700 1 -- 172.21.15.38:0/2570477618 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa57801e3b0 con 0x7fa580153cb0 2024-08-15T17:59:23.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.053+0000 7fa57cff9700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa56c060860 0x7fa56c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:23.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.053+0000 7fa57cff9700 1 -- 172.21.15.38:0/2570477618 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa57807cf90 con 0x7fa580153cb0 2024-08-15T17:59:23.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.053+0000 7fa584861700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa56c060860 0x7fa56c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:23.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.053+0000 7fa584861700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa56c060860 0x7fa56c062d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa574006fd0 tx=0x7fa574008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:23.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.065+0000 7fa57cff9700 1 -- 172.21.15.38:0/2570477618 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa5780ad480 con 0x7fa580153cb0 2024-08-15T17:59:23.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:23.269+0000 7fa586ac5700 1 -- 172.21.15.38:0/2570477618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa5800645c0 con 0x7fa580153cb0 2024-08-15T17:59:24.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.017+0000 7fa57cff9700 1 -- 172.21.15.38:0/2570477618 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa57801a0c0 con 0x7fa580153cb0 2024-08-15T17:59:24.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 -- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa56c060860 msgr2=0x7fa56c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa56c060860 0x7fa56c062d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa574006fd0 tx=0x7fa574008040 comp rx=0 tx=0).stop 2024-08-15T17:59:24.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 -- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa580153cb0 msgr2=0x7fa580152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa580153cb0 0x7fa580152c00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa578009fa0 tx=0x7fa578009990 comp rx=0 tx=0).stop 2024-08-15T17:59:24.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 -- 172.21.15.38:0/2570477618 shutdown_connections 2024-08-15T17:59:24.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa56c060860 0x7fa56c062d10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa580153cb0 0x7fa580152c00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5801538e0 0x7fa5801526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 --2- 172.21.15.38:0/2570477618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58006f180 0x7fa5801427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 -- 172.21.15.38:0/2570477618 >> 172.21.15.38:0/2570477618 conn(0x7fa580066e00 msgr2=0x7fa5801540d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:24.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 -- 172.21.15.38:0/2570477618 shutdown_connections 2024-08-15T17:59:24.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.021+0000 7fa586ac5700 1 -- 172.21.15.38:0/2570477618 wait complete. 2024-08-15T17:59:24.040 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:59:24.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 Processor -- start 2024-08-15T17:59:24.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 -- start start 2024-08-15T17:59:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa09c0abe80 0x7fa09c0ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa09c159700 0x7fa09c0686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa09c068be0 0x7fa09c0b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c15a8b0 con 0x7fa09c0abe80 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c0ae4e0 con 0x7fa09c159700 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c15ab20 con 0x7fa09c068be0 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa09c0abe80 0x7fa09c0ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa09c0abe80 0x7fa09c0ae9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41728/0 (socket says 172.21.15.38:41728) 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09bfff700 1 -- 172.21.15.38:0/1036240666 learned_addr learned my addr 172.21.15.38:0/1036240666 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09b7fe700 1 --2- 172.21.15.38:0/1036240666 >> v2:172.21.15.104:3300/0 conn(0x7fa09c159700 0x7fa09c0686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a0ec3700 1 --2- 172.21.15.38:0/1036240666 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa09c068be0 0x7fa09c0b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09bfff700 1 -- 172.21.15.38:0/1036240666 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa09c068be0 msgr2=0x7fa09c0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09bfff700 1 --2- 172.21.15.38:0/1036240666 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa09c068be0 0x7fa09c0b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09bfff700 1 -- 172.21.15.38:0/1036240666 >> v2:172.21.15.104:3300/0 conn(0x7fa09c159700 msgr2=0x7fa09c0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09bfff700 1 --2- 172.21.15.38:0/1036240666 >> v2:172.21.15.104:3300/0 conn(0x7fa09c159700 0x7fa09c0686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09bfff700 1 -- 172.21.15.38:0/1036240666 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c069030 con 0x7fa09c0abe80 2024-08-15T17:59:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09b7fe700 1 --2- 172.21.15.38:0/1036240666 >> v2:172.21.15.104:3300/0 conn(0x7fa09c159700 0x7fa09c0686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09bfff700 1 --2- 172.21.15.38:0/1036240666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0abe80 0x7fa09c0ae9e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa09c06d670 tx=0x7fa08c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25120ce4f3d19285 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:24.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09affd700 1 -- 172.21.15.38:0/1036240666 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00b640 con 0x7fa09c0abe80 2024-08-15T17:59:24.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09affd700 1 -- 172.21.15.38:0/1036240666 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c00b7a0 con 0x7fa09c0abe80 2024-08-15T17:59:24.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa09affd700 1 -- 172.21.15.38:0/1036240666 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00bb60 con 0x7fa09c0abe80 2024-08-15T17:59:24.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 -- 172.21.15.38:0/1036240666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0abe80 msgr2=0x7fa09c0ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 --2- 172.21.15.38:0/1036240666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0abe80 0x7fa09c0ae9e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa09c06d670 tx=0x7fa08c009250 comp rx=0 tx=0).stop 2024-08-15T17:59:24.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 -- 172.21.15.38:0/1036240666 shutdown_connections 2024-08-15T17:59:24.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 --2- 172.21.15.38:0/1036240666 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa09c068be0 0x7fa09c0b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 --2- 172.21.15.38:0/1036240666 >> v2:172.21.15.104:3300/0 conn(0x7fa09c159700 0x7fa09c0686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 --2- 172.21.15.38:0/1036240666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0abe80 0x7fa09c0ae9e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 -- 172.21.15.38:0/1036240666 >> 172.21.15.38:0/1036240666 conn(0x7fa09c06d870 msgr2=0x7fa09c06dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:24.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 -- 172.21.15.38:0/1036240666 shutdown_connections 2024-08-15T17:59:24.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.153+0000 7fa0a16c4700 1 -- 172.21.15.38:0/1036240666 wait complete. 2024-08-15T17:59:24.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a16c4700 1 Processor -- start 2024-08-15T17:59:24.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a16c4700 1 -- start start 2024-08-15T17:59:24.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a16c4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa09c068be0 0x7fa09c152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a16c4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0abe80 0x7fa09c15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a16c4700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa09c159700 0x7fa09c15cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a16c4700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c15b320 con 0x7fa09c0abe80 2024-08-15T17:59:24.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a16c4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c15f7b0 con 0x7fa09c068be0 2024-08-15T17:59:24.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a16c4700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c15fa90 con 0x7fa09c159700 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0abe80 0x7fa09c15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0abe80 0x7fa09c15ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41734/0 (socket says 172.21.15.38:41734) 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa09b7fe700 1 -- 172.21.15.38:0/2180315578 learned_addr learned my addr 172.21.15.38:0/2180315578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa09bfff700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa09c068be0 0x7fa09c152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a0ec3700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa09c159700 0x7fa09c15cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa09b7fe700 1 -- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa09c159700 msgr2=0x7fa09c15cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa09b7fe700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa09c159700 0x7fa09c15cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa09b7fe700 1 -- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa09c068be0 msgr2=0x7fa09c152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa09b7fe700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa09c068be0 0x7fa09c152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa09b7fe700 1 -- 172.21.15.38:0/2180315578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa084009a20 con 0x7fa09c0abe80 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa09b7fe700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0abe80 0x7fa09c15ecd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa08400c0c0 tx=0x7fa08400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa093fff700 1 -- 172.21.15.38:0/2180315578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08400cd60 con 0x7fa09c0abe80 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa093fff700 1 -- 172.21.15.38:0/2180315578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08400f040 con 0x7fa09c0abe80 2024-08-15T17:59:24.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa093fff700 1 -- 172.21.15.38:0/2180315578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084014690 con 0x7fa09c0abe80 2024-08-15T17:59:24.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a16c4700 1 -- 172.21.15.38:0/2180315578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08c0094f0 con 0x7fa09c0abe80 2024-08-15T17:59:24.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a16c4700 1 -- 172.21.15.38:0/2180315578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c15b030 con 0x7fa09c0abe80 2024-08-15T17:59:24.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa0a16c4700 1 -- 172.21.15.38:0/2180315578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c155fe0 con 0x7fa09c0abe80 2024-08-15T17:59:24.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa093fff700 1 -- 172.21.15.38:0/2180315578 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa084014a00 con 0x7fa09c0abe80 2024-08-15T17:59:24.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa093fff700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa07c060860 0x7fa07c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa093fff700 1 -- 172.21.15.38:0/2180315578 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa08407d9d0 con 0x7fa09c0abe80 2024-08-15T17:59:24.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.161+0000 7fa09bfff700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa07c060860 0x7fa07c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.169+0000 7fa09bfff700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa07c060860 0x7fa07c062d10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa09c06d670 tx=0x7fa08c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:24.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.173+0000 7fa093fff700 1 -- 172.21.15.38:0/2180315578 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa09c155fe0 con 0x7fa09c0abe80 2024-08-15T17:59:24.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.369+0000 7fa0a16c4700 1 -- 172.21.15.38:0/2180315578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa09c155fe0 con 0x7fa09c0abe80 2024-08-15T17:59:24.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.369+0000 7fa093fff700 1 -- 172.21.15.38:0/2180315578 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4594 (secure 0 0 0) 0x7fa09c155fe0 con 0x7fa09c0abe80 2024-08-15T17:59:24.373 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:24.373 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/373633111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":373633111},{"type":"v1","addr":"172.21.15.38:6833","nonce":373633111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/389886189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":389886189},{"type":"v1","addr":"172.21.15.38:6835","nonce":389886189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4981,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/875828080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":875828080},{"type":"v1","addr":"172.21.15.104:6835","nonce":875828080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:20.353438+0000","modified":"2024-08-15T17:59:24.015140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4999},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4999":{"gid":4999,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/1865762791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1865762791},{"type":"v1","addr":"172.21.15.104:6837","nonce":1865762791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-08-15T17:59:24.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 -- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa07c060860 msgr2=0x7fa07c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa07c060860 0x7fa07c062d10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa09c06d670 tx=0x7fa08c017000 comp rx=0 tx=0).stop 2024-08-15T17:59:24.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 -- 172.21.15.38:0/2180315578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0abe80 msgr2=0x7fa09c15ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0abe80 0x7fa09c15ecd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa08400c0c0 tx=0x7fa08400b040 comp rx=0 tx=0).stop 2024-08-15T17:59:24.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 -- 172.21.15.38:0/2180315578 shutdown_connections 2024-08-15T17:59:24.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fa07c060860 0x7fa07c062d10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa09c159700 0x7fa09c15cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0abe80 0x7fa09c15ecd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 --2- 172.21.15.38:0/2180315578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa09c068be0 0x7fa09c152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 -- 172.21.15.38:0/2180315578 >> 172.21.15.38:0/2180315578 conn(0x7fa09c06d870 msgr2=0x7fa09c0aa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:24.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 -- 172.21.15.38:0/2180315578 shutdown_connections 2024-08-15T17:59:24.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.373+0000 7fa0a16c4700 1 -- 172.21.15.38:0/2180315578 wait complete. 2024-08-15T17:59:24.377 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 15 2024-08-15T17:59:24.388 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:59:24.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 Processor -- start 2024-08-15T17:59:24.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 -- start start 2024-08-15T17:59:24.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68000cb560 0x7f6800161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68000cbaa0 con 0x7f6800154900 2024-08-15T17:59:24.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6800157630 con 0x7f68000b14a0 2024-08-15T17:59:24.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6800068790 con 0x7f68000cb560 2024-08-15T17:59:24.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67feffd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6800154900 0x7f68000b0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59208/0 (socket says 172.21.15.38:59208) 2024-08-15T17:59:24.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67feffd700 1 -- 172.21.15.38:0/1466442359 learned_addr learned my addr 172.21.15.38:0/1466442359 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:24.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67ff7fe700 1 --2- 172.21.15.38:0/1466442359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68000cb560 0x7f6800161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fdffb700 1 -- 172.21.15.38:0/1466442359 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811141165 0 0) 0x7f68000cbaa0 con 0x7f6800154900 2024-08-15T17:59:24.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fdffb700 1 -- 172.21.15.38:0/1466442359 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ec003570 con 0x7f6800154900 2024-08-15T17:59:24.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67ff7fe700 1 -- 172.21.15.38:0/1466442359 >> v1:172.21.15.104:6789/0 conn(0x7f68000b14a0 legacy=0x7f68000cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67ff7fe700 1 -- 172.21.15.38:0/1466442359 >> v1:172.21.15.38:6789/0 conn(0x7f6800154900 legacy=0x7f68000b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fdffb700 1 -- 172.21.15.38:0/1466442359 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745461092 0 0) 0x7f6800157630 con 0x7f68000b14a0 2024-08-15T17:59:24.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67ff7fe700 1 -- 172.21.15.38:0/1466442359 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800157910 con 0x7f68000cb560 2024-08-15T17:59:24.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67ff7fe700 1 --2- 172.21.15.38:0/1466442359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68000cb560 0x7f6800161ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009e90 tx=0x7f67f8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8c6d99daad98c66 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fdffb700 1 -- 172.21.15.38:0/1466442359 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800e070 con 0x7f68000cb560 2024-08-15T17:59:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fdffb700 1 -- 172.21.15.38:0/1466442359 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f800bc30 con 0x7f68000cb560 2024-08-15T17:59:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fdffb700 1 -- 172.21.15.38:0/1466442359 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8012430 con 0x7f68000cb560 2024-08-15T17:59:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 -- 172.21.15.38:0/1466442359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68000cb560 msgr2=0x7f6800161ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 --2- 172.21.15.38:0/1466442359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68000cb560 0x7f6800161ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009e90 tx=0x7f67f8009780 comp rx=0 tx=0).stop 2024-08-15T17:59:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 -- 172.21.15.38:0/1466442359 shutdown_connections 2024-08-15T17:59:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 --2- 172.21.15.38:0/1466442359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68000cb560 0x7f6800161ad0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 -- 172.21.15.38:0/1466442359 >> 172.21.15.38:0/1466442359 conn(0x7f68000678d0 msgr2=0x7f6800067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 -- 172.21.15.38:0/1466442359 shutdown_connections 2024-08-15T17:59:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.493+0000 7f67fffff700 1 -- 172.21.15.38:0/1466442359 wait complete. 2024-08-15T17:59:24.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67fffff700 1 Processor -- start 2024-08-15T17:59:24.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67fffff700 1 -- start start 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67fffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000b14a0 0x7f680014f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67fffff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68000cb560 0x7f680014de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67fffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6800154900 0x7f6800063060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67fffff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680015b020 con 0x7f68000b14a0 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67fffff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680016abe0 con 0x7f6800154900 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67fffff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680016aec0 con 0x7f68000cb560 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67feffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000b14a0 0x7f680014f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67feffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000b14a0 0x7f680014f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41748/0 (socket says 172.21.15.38:41748) 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67feffd700 1 -- 172.21.15.38:0/3476792468 learned_addr learned my addr 172.21.15.38:0/3476792468 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67fe7fc700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68000cb560 0x7f680014de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67ff7fe700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6800154900 0x7f6800063060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67feffd700 1 -- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68000cb560 msgr2=0x7f680014de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67feffd700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68000cb560 0x7f680014de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67feffd700 1 -- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6800154900 msgr2=0x7f6800063060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67feffd700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6800154900 0x7f6800063060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67feffd700 1 -- 172.21.15.38:0/3476792468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f8009a20 con 0x7f68000b14a0 2024-08-15T17:59:24.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67feffd700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000b14a0 0x7f680014f200 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f67f00011e0 tx=0x7f67f0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:24.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67f6ffd700 1 -- 172.21.15.38:0/3476792468 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0005b30 con 0x7f68000b14a0 2024-08-15T17:59:24.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67f6ffd700 1 -- 172.21.15.38:0/3476792468 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f0005c90 con 0x7f68000b14a0 2024-08-15T17:59:24.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67fffff700 1 -- 172.21.15.38:0/3476792468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680016b1a0 con 0x7f68000b14a0 2024-08-15T17:59:24.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67f6ffd700 1 -- 172.21.15.38:0/3476792468 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0007320 con 0x7f68000b14a0 2024-08-15T17:59:24.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.497+0000 7f67fffff700 1 -- 172.21.15.38:0/3476792468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680014e340 con 0x7f68000b14a0 2024-08-15T17:59:24.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.501+0000 7f67f6ffd700 1 -- 172.21.15.38:0/3476792468 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f67f0007690 con 0x7f68000b14a0 2024-08-15T17:59:24.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.501+0000 7f67fffff700 1 -- 172.21.15.38:0/3476792468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68000643a0 con 0x7f68000b14a0 2024-08-15T17:59:24.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.501+0000 7f67f6ffd700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f67e0060890 0x7f67e0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.501+0000 7f67f6ffd700 1 -- 172.21.15.38:0/3476792468 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f67f007beb0 con 0x7f68000b14a0 2024-08-15T17:59:24.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.501+0000 7f67fe7fc700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f67e0060890 0x7f67e0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.505+0000 7f67fe7fc700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f67e0060890 0x7f67e0062d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f680006b230 tx=0x7f67e8007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:24.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.509+0000 7f67f6ffd700 1 -- 172.21.15.38:0/3476792468 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f67f004c300 con 0x7f68000b14a0 2024-08-15T17:59:24.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.677+0000 7f67fffff700 1 -- 172.21.15.38:0/3476792468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68000aded0 con 0x7f68000b14a0 2024-08-15T17:59:24.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.677+0000 7f67f6ffd700 1 -- 172.21.15.38:0/3476792468 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f67f004f920 con 0x7f68000b14a0 2024-08-15T17:59:24.682 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:24.682 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":32,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:59:21.991939+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:59:18.793510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:59:19.328139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:59:24.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 -- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f67e0060890 msgr2=0x7f67e0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f67e0060890 0x7f67e0062d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f680006b230 tx=0x7f67e8007d30 comp rx=0 tx=0).stop 2024-08-15T17:59:24.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 -- 172.21.15.38:0/3476792468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000b14a0 msgr2=0x7f680014f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000b14a0 0x7f680014f200 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f67f00011e0 tx=0x7f67f0007df0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 -- 172.21.15.38:0/3476792468 shutdown_connections 2024-08-15T17:59:24.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f67e0060890 0x7f67e0062d40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6800154900 0x7f6800063060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68000cb560 0x7f680014de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 --2- 172.21.15.38:0/3476792468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000b14a0 0x7f680014f200 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 -- 172.21.15.38:0/3476792468 >> 172.21.15.38:0/3476792468 conn(0x7f68000678d0 msgr2=0x7f6800153900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:24.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 -- 172.21.15.38:0/3476792468 shutdown_connections 2024-08-15T17:59:24.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.681+0000 7f67fffff700 1 -- 172.21.15.38:0/3476792468 wait complete. 2024-08-15T17:59:24.696 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-08-15T17:59:24.697 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:59:24.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9dad6bf700 1 Processor -- start 2024-08-15T17:59:24.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9dad6bf700 1 -- start start 2024-08-15T17:59:24.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9dad6bf700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9da806e9d0 0x7f9da806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9dad6bf700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9da80ab540 0x7f9da80b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9dad6bf700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9da8155010 0x7f9da8155490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9dad6bf700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da81559d0 con 0x7f9da806e9d0 2024-08-15T17:59:24.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9dad6bf700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da80aba70 con 0x7f9da80ab540 2024-08-15T17:59:24.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9dad6bf700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80abd50 con 0x7f9da8155010 2024-08-15T17:59:24.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9da7fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9da806e9d0 0x7f9da806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9da7fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9da806e9d0 0x7f9da806eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41754/0 (socket says 172.21.15.38:41754) 2024-08-15T17:59:24.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9da7fff700 1 -- 172.21.15.38:0/3037303844 learned_addr learned my addr 172.21.15.38:0/3037303844 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:24.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9da77fe700 1 --2- 172.21.15.38:0/3037303844 >> v2:172.21.15.104:3300/0 conn(0x7f9da80ab540 0x7f9da80b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9dacebe700 1 --2- 172.21.15.38:0/3037303844 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9da8155010 0x7f9da8155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9da77fe700 1 -- 172.21.15.38:0/3037303844 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9da8155010 msgr2=0x7f9da8155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:24.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9da77fe700 1 --2- 172.21.15.38:0/3037303844 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9da8155010 0x7f9da8155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9da77fe700 1 -- 172.21.15.38:0/3037303844 >> v2:172.21.15.38:3300/0 conn(0x7f9da806e9d0 msgr2=0x7f9da806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:24.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9da77fe700 1 --2- 172.21.15.38:0/3037303844 >> v2:172.21.15.38:3300/0 conn(0x7f9da806e9d0 0x7f9da806eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9da77fe700 1 -- 172.21.15.38:0/3037303844 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da80ac030 con 0x7f9da80ab540 2024-08-15T17:59:24.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.805+0000 7f9da7fff700 1 --2- 172.21.15.38:0/3037303844 >> v2:172.21.15.38:3300/0 conn(0x7f9da806e9d0 0x7f9da806eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:24.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9da77fe700 1 --2- 172.21.15.38:0/3037303844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9da80ab540 0x7f9da80b1320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9da80ae820 tx=0x7f9d90009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12f90e55ee547bf9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9da6ffd700 1 -- 172.21.15.38:0/3037303844 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000e070 con 0x7f9da80ab540 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9da6ffd700 1 -- 172.21.15.38:0/3037303844 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9000bab0 con 0x7f9da80ab540 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9da6ffd700 1 -- 172.21.15.38:0/3037303844 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000be70 con 0x7f9da80ab540 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 -- 172.21.15.38:0/3037303844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9da80ab540 msgr2=0x7f9da80b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 --2- 172.21.15.38:0/3037303844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9da80ab540 0x7f9da80b1320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9da80ae820 tx=0x7f9d90009780 comp rx=0 tx=0).stop 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 -- 172.21.15.38:0/3037303844 shutdown_connections 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 --2- 172.21.15.38:0/3037303844 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9da8155010 0x7f9da8155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 --2- 172.21.15.38:0/3037303844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9da80ab540 0x7f9da80b1320 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 --2- 172.21.15.38:0/3037303844 >> v2:172.21.15.38:3300/0 conn(0x7f9da806e9d0 0x7f9da806eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 -- 172.21.15.38:0/3037303844 >> 172.21.15.38:0/3037303844 conn(0x7f9da80678d0 msgr2=0x7f9da8067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 -- 172.21.15.38:0/3037303844 shutdown_connections 2024-08-15T17:59:24.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 -- 172.21.15.38:0/3037303844 wait complete. 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 Processor -- start 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 -- start start 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9da806e9d0 0x7f9da80a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9da80ab540 0x7f9da814b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da8155010 0x7f9da8156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80aaca0 con 0x7f9da8155010 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80ab130 con 0x7f9da80ab540 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.809+0000 7f9dad6bf700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80aa090 con 0x7f9da806e9d0 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dacebe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da8155010 0x7f9da8156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dacebe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da8155010 0x7f9da8156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41770/0 (socket says 172.21.15.38:41770) 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dacebe700 1 -- 172.21.15.38:0/2767682459 learned_addr learned my addr 172.21.15.38:0/2767682459 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:24.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9da77fe700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9da80ab540 0x7f9da814b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9da7fff700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9da806e9d0 0x7f9da80a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dacebe700 1 -- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9da806e9d0 msgr2=0x7f9da80a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dacebe700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9da806e9d0 0x7f9da80a9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dacebe700 1 -- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9da80ab540 msgr2=0x7f9da814b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:24.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dacebe700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9da80ab540 0x7f9da814b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:24.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dacebe700 1 -- 172.21.15.38:0/2767682459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d980094f0 con 0x7f9da8155010 2024-08-15T17:59:24.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9da77fe700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9da80ab540 0x7f9da814b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:24.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dacebe700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da8155010 0x7f9da8156a10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9da000e8d0 tx=0x7f9da00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:24.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9d9ffff700 1 -- 172.21.15.38:0/2767682459 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0013680 con 0x7f9da8155010 2024-08-15T17:59:24.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9d9ffff700 1 -- 172.21.15.38:0/2767682459 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da00137e0 con 0x7f9da8155010 2024-08-15T17:59:24.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dad6bf700 1 -- 172.21.15.38:0/2767682459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d90009a20 con 0x7f9da8155010 2024-08-15T17:59:24.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9d9ffff700 1 -- 172.21.15.38:0/2767682459 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0013ba0 con 0x7f9da8155010 2024-08-15T17:59:24.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dad6bf700 1 -- 172.21.15.38:0/2767682459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da80aa650 con 0x7f9da8155010 2024-08-15T17:59:24.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.813+0000 7f9dad6bf700 1 -- 172.21.15.38:0/2767682459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da80aae10 con 0x7f9da8155010 2024-08-15T17:59:24.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.817+0000 7f9d9ffff700 1 -- 172.21.15.38:0/2767682459 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9da00123d0 con 0x7f9da8155010 2024-08-15T17:59:24.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.817+0000 7f9d9ffff700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9d88060860 0x7f9d88062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:24.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.817+0000 7f9d9ffff700 1 -- 172.21.15.38:0/2767682459 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9da007cc70 con 0x7f9da8155010 2024-08-15T17:59:24.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.817+0000 7f9da7fff700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9d88060860 0x7f9d88062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:24.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.817+0000 7f9da7fff700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9d88060860 0x7f9d88062d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9d98006d20 tx=0x7f9d98009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:24.817 INFO:tasks.ceph.mds.d.smithi104.stderr:2024-08-15T17:59:24.814+0000 7f199d96b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T17:59:24.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.821+0000 7f9d9ffff700 1 -- 172.21.15.38:0/2767682459 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9da004d0c0 con 0x7f9da8155010 2024-08-15T17:59:24.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:24.985+0000 7f9dad6bf700 1 -- 172.21.15.38:0/2767682459 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9da80643a0 con 0x7f9d88060860 2024-08-15T17:59:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.001+0000 7f9d9ffff700 1 -- 172.21.15.38:0/2767682459 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264608 (secure 0 0 0) 0x7f9da80643a0 con 0x7f9d88060860 2024-08-15T17:59:25.005 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:25.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 -- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9d88060860 msgr2=0x7f9d88062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9d88060860 0x7f9d88062d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9d98006d20 tx=0x7f9d98009250 comp rx=0 tx=0).stop 2024-08-15T17:59:25.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 -- 172.21.15.38:0/2767682459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da8155010 msgr2=0x7f9da8156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da8155010 0x7f9da8156a10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9da000e8d0 tx=0x7f9da00091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 -- 172.21.15.38:0/2767682459 shutdown_connections 2024-08-15T17:59:25.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9d88060860 0x7f9d88062d10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da8155010 0x7f9da8156a10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9da80ab540 0x7f9da814b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 --2- 172.21.15.38:0/2767682459 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9da806e9d0 0x7f9da80a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 -- 172.21.15.38:0/2767682459 >> 172.21.15.38:0/2767682459 conn(0x7f9da80678d0 msgr2=0x7f9da8069160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:25.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 -- 172.21.15.38:0/2767682459 shutdown_connections 2024-08-15T17:59:25.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.005+0000 7f9dad6bf700 1 -- 172.21.15.38:0/2767682459 wait complete. 2024-08-15T17:59:25.008 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-08-15T17:59:25.020 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-08-15T17:59:24.184619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2399404,"kb_used_data":3892,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752575316,"statfs":{"total":773094113280,"available":770637123584,"internally_reserved":0,"allocated":3985408,"data_stored":2228284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220040"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016143+0000","last_change":"2024-08-15T17:59:21.009226+0000","last_active":"2024-08-15T17:59:21.016143+0000","last_peered":"2024-08-15T17:59:21.016143+0000","last_clean":"2024-08-15T17:59:21.016143+0000","last_became_active":"2024-08-15T17:59:21.007242+0000","last_became_peered":"2024-08-15T17:59:21.007242+0000","last_unstale":"2024-08-15T17:59:21.016143+0000","last_undegraded":"2024-08-15T17:59:21.016143+0000","last_fullsized":"2024-08-15T17:59:21.016143+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:52:47.789349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.006240+0000","last_change":"2024-08-15T17:59:21.006240+0000","last_active":"2024-08-15T17:59:21.006240+0000","last_peered":"2024-08-15T17:59:21.006240+0000","last_clean":"2024-08-15T17:59:21.006240+0000","last_became_active":"2024-08-15T17:59:21.005175+0000","last_became_peered":"2024-08-15T17:59:21.005175+0000","last_unstale":"2024-08-15T17:59:21.006240+0000","last_undegraded":"2024-08-15T17:59:21.006240+0000","last_fullsized":"2024-08-15T17:59:21.006240+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:42:51.216038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004133+0000","last_change":"2024-08-15T17:59:21.004133+0000","last_active":"2024-08-15T17:59:21.004133+0000","last_peered":"2024-08-15T17:59:21.004133+0000","last_clean":"2024-08-15T17:59:21.004133+0000","last_became_active":"2024-08-15T17:59:21.003851+0000","last_became_peered":"2024-08-15T17:59:21.003851+0000","last_unstale":"2024-08-15T17:59:21.004133+0000","last_undegraded":"2024-08-15T17:59:21.004133+0000","last_fullsized":"2024-08-15T17:59:21.004133+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:33:33.720588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016619+0000","last_change":"2024-08-15T17:59:21.008278+0000","last_active":"2024-08-15T17:59:21.016619+0000","last_peered":"2024-08-15T17:59:21.016619+0000","last_clean":"2024-08-15T17:59:21.016619+0000","last_became_active":"2024-08-15T17:59:21.007966+0000","last_became_peered":"2024-08-15T17:59:21.007966+0000","last_unstale":"2024-08-15T17:59:21.016619+0000","last_undegraded":"2024-08-15T17:59:21.016619+0000","last_fullsized":"2024-08-15T17:59:21.016619+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:15:12.056272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004929+0000","last_change":"2024-08-15T17:59:21.004929+0000","last_active":"2024-08-15T17:59:21.004929+0000","last_peered":"2024-08-15T17:59:21.004929+0000","last_clean":"2024-08-15T17:59:21.004929+0000","last_became_active":"2024-08-15T17:59:21.004600+0000","last_became_peered":"2024-08-15T17:59:21.004600+0000","last_unstale":"2024-08-15T17:59:21.004929+0000","last_undegraded":"2024-08-15T17:59:21.004929+0000","last_fullsized":"2024-08-15T17:59:21.004929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:43:53.451504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.007284+0000","last_change":"2024-08-15T17:59:21.007284+0000","last_active":"2024-08-15T17:59:21.007284+0000","last_peered":"2024-08-15T17:59:21.007284+0000","last_clean":"2024-08-15T17:59:21.007284+0000","last_became_active":"2024-08-15T17:59:21.006962+0000","last_became_peered":"2024-08-15T17:59:21.006962+0000","last_unstale":"2024-08-15T17:59:21.007284+0000","last_undegraded":"2024-08-15T17:59:21.007284+0000","last_fullsized":"2024-08-15T17:59:21.007284+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:34:38.302879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.010719+0000","last_change":"2024-08-15T17:59:21.010719+0000","last_active":"2024-08-15T17:59:21.010719+0000","last_peered":"2024-08-15T17:59:21.010719+0000","last_clean":"2024-08-15T17:59:21.010719+0000","last_became_active":"2024-08-15T17:59:21.010425+0000","last_became_peered":"2024-08-15T17:59:21.010425+0000","last_unstale":"2024-08-15T17:59:21.010719+0000","last_undegraded":"2024-08-15T17:59:21.010719+0000","last_fullsized":"2024-08-15T17:59:21.010719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:27:52.825284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.008627+0000","last_change":"2024-08-15T17:59:21.008627+0000","last_active":"2024-08-15T17:59:21.008627+0000","last_peered":"2024-08-15T17:59:21.008627+0000","last_clean":"2024-08-15T17:59:21.008627+0000","last_became_active":"2024-08-15T17:59:21.008366+0000","last_became_peered":"2024-08-15T17:59:21.008366+0000","last_unstale":"2024-08-15T17:59:21.008627+0000","last_undegraded":"2024-08-15T17:59:21.008627+0000","last_fullsized":"2024-08-15T17:59:21.008627+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:17:19.740984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004751+0000","last_change":"2024-08-15T17:59:21.004751+0000","last_active":"2024-08-15T17:59:21.004751+0000","last_peered":"2024-08-15T17:59:21.004751+0000","last_clean":"2024-08-15T17:59:21.004751+0000","last_became_active":"2024-08-15T17:59:21.004215+0000","last_became_peered":"2024-08-15T17:59:21.004215+0000","last_unstale":"2024-08-15T17:59:21.004751+0000","last_undegraded":"2024-08-15T17:59:21.004751+0000","last_fullsized":"2024-08-15T17:59:21.004751+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:09:21.391162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016211+0000","last_change":"2024-08-15T17:59:21.011249+0000","last_active":"2024-08-15T17:59:21.016211+0000","last_peered":"2024-08-15T17:59:21.016211+0000","last_clean":"2024-08-15T17:59:21.016211+0000","last_became_active":"2024-08-15T17:59:21.010935+0000","last_became_peered":"2024-08-15T17:59:21.010935+0000","last_unstale":"2024-08-15T17:59:21.016211+0000","last_undegraded":"2024-08-15T17:59:21.016211+0000","last_fullsized":"2024-08-15T17:59:21.016211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:34:50.258303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004941+0000","last_change":"2024-08-15T17:59:21.004941+0000","last_active":"2024-08-15T17:59:21.004941+0000","last_peered":"2024-08-15T17:59:21.004941+0000","last_clean":"2024-08-15T17:59:21.004941+0000","last_became_active":"2024-08-15T17:59:21.004665+0000","last_became_peered":"2024-08-15T17:59:21.004665+0000","last_unstale":"2024-08-15T17:59:21.004941+0000","last_undegraded":"2024-08-15T17:59:21.004941+0000","last_fullsized":"2024-08-15T17:59:21.004941+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:05:39.941247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.001954+0000","last_change":"2024-08-15T17:59:21.001954+0000","last_active":"2024-08-15T17:59:21.001954+0000","last_peered":"2024-08-15T17:59:21.001954+0000","last_clean":"2024-08-15T17:59:21.001954+0000","last_became_active":"2024-08-15T17:59:21.001503+0000","last_became_peered":"2024-08-15T17:59:21.001503+0000","last_unstale":"2024-08-15T17:59:21.001954+0000","last_undegraded":"2024-08-15T17:59:21.001954+0000","last_fullsized":"2024-08-15T17:59:21.001954+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:20:04.815893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004865+0000","last_change":"2024-08-15T17:59:21.004865+0000","last_active":"2024-08-15T17:59:21.004865+0000","last_peered":"2024-08-15T17:59:21.004865+0000","last_clean":"2024-08-15T17:59:21.004865+0000","last_became_active":"2024-08-15T17:59:21.004538+0000","last_became_peered":"2024-08-15T17:59:21.004538+0000","last_unstale":"2024-08-15T17:59:21.004865+0000","last_undegraded":"2024-08-15T17:59:21.004865+0000","last_fullsized":"2024-08-15T17:59:21.004865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:52:05.703059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.010456+0000","last_change":"2024-08-15T17:59:21.010456+0000","last_active":"2024-08-15T17:59:21.010456+0000","last_peered":"2024-08-15T17:59:21.010456+0000","last_clean":"2024-08-15T17:59:21.010456+0000","last_became_active":"2024-08-15T17:59:21.010081+0000","last_became_peered":"2024-08-15T17:59:21.010081+0000","last_unstale":"2024-08-15T17:59:21.010456+0000","last_undegraded":"2024-08-15T17:59:21.010456+0000","last_fullsized":"2024-08-15T17:59:21.010456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:22:59.034828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.011051+0000","last_change":"2024-08-15T17:59:21.011051+0000","last_active":"2024-08-15T17:59:21.011051+0000","last_peered":"2024-08-15T17:59:21.011051+0000","last_clean":"2024-08-15T17:59:21.011051+0000","last_became_active":"2024-08-15T17:59:21.010785+0000","last_became_peered":"2024-08-15T17:59:21.010785+0000","last_unstale":"2024-08-15T17:59:21.011051+0000","last_undegraded":"2024-08-15T17:59:21.011051+0000","last_fullsized":"2024-08-15T17:59:21.011051+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:41:41.362879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003364+0000","last_change":"2024-08-15T17:59:21.003364+0000","last_active":"2024-08-15T17:59:21.003364+0000","last_peered":"2024-08-15T17:59:21.003364+0000","last_clean":"2024-08-15T17:59:21.003364+0000","last_became_active":"2024-08-15T17:59:21.002797+0000","last_became_peered":"2024-08-15T17:59:21.002797+0000","last_unstale":"2024-08-15T17:59:21.003364+0000","last_undegraded":"2024-08-15T17:59:21.003364+0000","last_fullsized":"2024-08-15T17:59:21.003364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:24:53.455619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.009953+0000","last_change":"2024-08-15T17:59:21.009953+0000","last_active":"2024-08-15T17:59:21.009953+0000","last_peered":"2024-08-15T17:59:21.009953+0000","last_clean":"2024-08-15T17:59:21.009953+0000","last_became_active":"2024-08-15T17:59:21.009698+0000","last_became_peered":"2024-08-15T17:59:21.009698+0000","last_unstale":"2024-08-15T17:59:21.009953+0000","last_undegraded":"2024-08-15T17:59:21.009953+0000","last_fullsized":"2024-08-15T17:59:21.009953+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:58:36.501838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.006436+0000","last_change":"2024-08-15T17:59:21.006436+0000","last_active":"2024-08-15T17:59:21.006436+0000","last_peered":"2024-08-15T17:59:21.006436+0000","last_clean":"2024-08-15T17:59:21.006436+0000","last_became_active":"2024-08-15T17:59:21.005550+0000","last_became_peered":"2024-08-15T17:59:21.005550+0000","last_unstale":"2024-08-15T17:59:21.006436+0000","last_undegraded":"2024-08-15T17:59:21.006436+0000","last_fullsized":"2024-08-15T17:59:21.006436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:23:10.592319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.005895+0000","last_change":"2024-08-15T17:59:21.005895+0000","last_active":"2024-08-15T17:59:21.005895+0000","last_peered":"2024-08-15T17:59:21.005895+0000","last_clean":"2024-08-15T17:59:21.005895+0000","last_became_active":"2024-08-15T17:59:21.005612+0000","last_became_peered":"2024-08-15T17:59:21.005612+0000","last_unstale":"2024-08-15T17:59:21.005895+0000","last_undegraded":"2024-08-15T17:59:21.005895+0000","last_fullsized":"2024-08-15T17:59:21.005895+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:20:04.815893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.008445+0000","last_change":"2024-08-15T17:59:21.008445+0000","last_active":"2024-08-15T17:59:21.008445+0000","last_peered":"2024-08-15T17:59:21.008445+0000","last_clean":"2024-08-15T17:59:21.008445+0000","last_became_active":"2024-08-15T17:59:21.008154+0000","last_became_peered":"2024-08-15T17:59:21.008154+0000","last_unstale":"2024-08-15T17:59:21.008445+0000","last_undegraded":"2024-08-15T17:59:21.008445+0000","last_fullsized":"2024-08-15T17:59:21.008445+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:29:18.099835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.996839+0000","last_change":"2024-08-15T17:59:20.996839+0000","last_active":"2024-08-15T17:59:20.996839+0000","last_peered":"2024-08-15T17:59:20.996839+0000","last_clean":"2024-08-15T17:59:20.996839+0000","last_became_active":"2024-08-15T17:59:20.996480+0000","last_became_peered":"2024-08-15T17:59:20.996480+0000","last_unstale":"2024-08-15T17:59:20.996839+0000","last_undegraded":"2024-08-15T17:59:20.996839+0000","last_fullsized":"2024-08-15T17:59:20.996839+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:14:54.232264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004296+0000","last_change":"2024-08-15T17:59:21.004296+0000","last_active":"2024-08-15T17:59:21.004296+0000","last_peered":"2024-08-15T17:59:21.004296+0000","last_clean":"2024-08-15T17:59:21.004296+0000","last_became_active":"2024-08-15T17:59:21.003472+0000","last_became_peered":"2024-08-15T17:59:21.003472+0000","last_unstale":"2024-08-15T17:59:21.004296+0000","last_undegraded":"2024-08-15T17:59:21.004296+0000","last_fullsized":"2024-08-15T17:59:21.004296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:48:44.709713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004532+0000","last_change":"2024-08-15T17:59:20.996638+0000","last_active":"2024-08-15T17:59:21.004532+0000","last_peered":"2024-08-15T17:59:21.004532+0000","last_clean":"2024-08-15T17:59:21.004532+0000","last_became_active":"2024-08-15T17:59:20.996332+0000","last_became_peered":"2024-08-15T17:59:20.996332+0000","last_unstale":"2024-08-15T17:59:21.004532+0000","last_undegraded":"2024-08-15T17:59:21.004532+0000","last_fullsized":"2024-08-15T17:59:21.004532+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:15:05.424547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.008964+0000","last_change":"2024-08-15T17:59:21.008964+0000","last_active":"2024-08-15T17:59:21.008964+0000","last_peered":"2024-08-15T17:59:21.008964+0000","last_clean":"2024-08-15T17:59:21.008964+0000","last_became_active":"2024-08-15T17:59:21.008709+0000","last_became_peered":"2024-08-15T17:59:21.008709+0000","last_unstale":"2024-08-15T17:59:21.008964+0000","last_undegraded":"2024-08-15T17:59:21.008964+0000","last_fullsized":"2024-08-15T17:59:21.008964+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:46:14.773091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016071+0000","last_change":"2024-08-15T17:59:21.008310+0000","last_active":"2024-08-15T17:59:21.016071+0000","last_peered":"2024-08-15T17:59:21.016071+0000","last_clean":"2024-08-15T17:59:21.016071+0000","last_became_active":"2024-08-15T17:59:21.007584+0000","last_became_peered":"2024-08-15T17:59:21.007584+0000","last_unstale":"2024-08-15T17:59:21.016071+0000","last_undegraded":"2024-08-15T17:59:21.016071+0000","last_fullsized":"2024-08-15T17:59:21.016071+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:21:08.329258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003903+0000","last_change":"2024-08-15T17:59:21.003903+0000","last_active":"2024-08-15T17:59:21.003903+0000","last_peered":"2024-08-15T17:59:21.003903+0000","last_clean":"2024-08-15T17:59:21.003903+0000","last_became_active":"2024-08-15T17:59:21.003517+0000","last_became_peered":"2024-08-15T17:59:21.003517+0000","last_unstale":"2024-08-15T17:59:21.003903+0000","last_undegraded":"2024-08-15T17:59:21.003903+0000","last_fullsized":"2024-08-15T17:59:21.003903+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:33:33.720588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016179+0000","last_change":"2024-08-15T17:59:21.011077+0000","last_active":"2024-08-15T17:59:21.016179+0000","last_peered":"2024-08-15T17:59:21.016179+0000","last_clean":"2024-08-15T17:59:21.016179+0000","last_became_active":"2024-08-15T17:59:21.008866+0000","last_became_peered":"2024-08-15T17:59:21.008866+0000","last_unstale":"2024-08-15T17:59:21.016179+0000","last_undegraded":"2024-08-15T17:59:21.016179+0000","last_fullsized":"2024-08-15T17:59:21.016179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:24:33.139116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004990+0000","last_change":"2024-08-15T17:59:21.004990+0000","last_active":"2024-08-15T17:59:21.004990+0000","last_peered":"2024-08-15T17:59:21.004990+0000","last_clean":"2024-08-15T17:59:21.004990+0000","last_became_active":"2024-08-15T17:59:21.004702+0000","last_became_peered":"2024-08-15T17:59:21.004702+0000","last_unstale":"2024-08-15T17:59:21.004990+0000","last_undegraded":"2024-08-15T17:59:21.004990+0000","last_fullsized":"2024-08-15T17:59:21.004990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:00:28.528787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003768+0000","last_change":"2024-08-15T17:59:21.003768+0000","last_active":"2024-08-15T17:59:21.003768+0000","last_peered":"2024-08-15T17:59:21.003768+0000","last_clean":"2024-08-15T17:59:21.003768+0000","last_became_active":"2024-08-15T17:59:21.003400+0000","last_became_peered":"2024-08-15T17:59:21.003400+0000","last_unstale":"2024-08-15T17:59:21.003768+0000","last_undegraded":"2024-08-15T17:59:21.003768+0000","last_fullsized":"2024-08-15T17:59:21.003768+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:52:47.789349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.008083+0000","last_change":"2024-08-15T17:59:21.008083+0000","last_active":"2024-08-15T17:59:21.008083+0000","last_peered":"2024-08-15T17:59:21.008083+0000","last_clean":"2024-08-15T17:59:21.008083+0000","last_became_active":"2024-08-15T17:59:21.007767+0000","last_became_peered":"2024-08-15T17:59:21.007767+0000","last_unstale":"2024-08-15T17:59:21.008083+0000","last_undegraded":"2024-08-15T17:59:21.008083+0000","last_fullsized":"2024-08-15T17:59:21.008083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:31:39.516975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016181+0000","last_change":"2024-08-15T17:59:21.010373+0000","last_active":"2024-08-15T17:59:21.016181+0000","last_peered":"2024-08-15T17:59:21.016181+0000","last_clean":"2024-08-15T17:59:21.016181+0000","last_became_active":"2024-08-15T17:59:21.010073+0000","last_became_peered":"2024-08-15T17:59:21.010073+0000","last_unstale":"2024-08-15T17:59:21.016181+0000","last_undegraded":"2024-08-15T17:59:21.016181+0000","last_fullsized":"2024-08-15T17:59:21.016181+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:48:44.709713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003043+0000","last_change":"2024-08-15T17:59:21.003043+0000","last_active":"2024-08-15T17:59:21.003043+0000","last_peered":"2024-08-15T17:59:21.003043+0000","last_clean":"2024-08-15T17:59:21.003043+0000","last_became_active":"2024-08-15T17:59:21.002267+0000","last_became_peered":"2024-08-15T17:59:21.002267+0000","last_unstale":"2024-08-15T17:59:21.003043+0000","last_undegraded":"2024-08-15T17:59:21.003043+0000","last_fullsized":"2024-08-15T17:59:21.003043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:18:33.342879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004423+0000","last_change":"2024-08-15T17:59:20.996955+0000","last_active":"2024-08-15T17:59:21.004423+0000","last_peered":"2024-08-15T17:59:21.004423+0000","last_clean":"2024-08-15T17:59:21.004423+0000","last_became_active":"2024-08-15T17:59:20.996652+0000","last_became_peered":"2024-08-15T17:59:20.996652+0000","last_unstale":"2024-08-15T17:59:21.004423+0000","last_undegraded":"2024-08-15T17:59:21.004423+0000","last_fullsized":"2024-08-15T17:59:21.004423+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:41.005396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.017799+0000","last_change":"2024-08-15T17:59:19.992199+0000","last_active":"2024-08-15T17:59:21.017799+0000","last_peered":"2024-08-15T17:59:21.017799+0000","last_clean":"2024-08-15T17:59:21.017799+0000","last_became_active":"2024-08-15T17:59:19.991914+0000","last_became_peered":"2024-08-15T17:59:19.991914+0000","last_unstale":"2024-08-15T17:59:21.017799+0000","last_undegraded":"2024-08-15T17:59:21.017799+0000","last_fullsized":"2024-08-15T17:59:21.017799+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:29:17.090193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.013746+0000","last_change":"2024-08-15T17:59:21.000563+0000","last_active":"2024-08-15T17:59:21.013746+0000","last_peered":"2024-08-15T17:59:21.013746+0000","last_clean":"2024-08-15T17:59:21.013746+0000","last_became_active":"2024-08-15T17:59:20.999966+0000","last_became_peered":"2024-08-15T17:59:20.999966+0000","last_unstale":"2024-08-15T17:59:21.013746+0000","last_undegraded":"2024-08-15T17:59:21.013746+0000","last_fullsized":"2024-08-15T17:59:21.013746+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:30:10.217672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.017920+0000","last_change":"2024-08-15T17:59:20.006977+0000","last_active":"2024-08-15T17:59:21.017920+0000","last_peered":"2024-08-15T17:59:21.017920+0000","last_clean":"2024-08-15T17:59:21.017920+0000","last_became_active":"2024-08-15T17:59:20.006674+0000","last_became_peered":"2024-08-15T17:59:20.006674+0000","last_unstale":"2024-08-15T17:59:21.017920+0000","last_undegraded":"2024-08-15T17:59:21.017920+0000","last_fullsized":"2024-08-15T17:59:21.017920+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:02:24.732030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003882+0000","last_change":"2024-08-15T17:59:21.003882+0000","last_active":"2024-08-15T17:59:21.003882+0000","last_peered":"2024-08-15T17:59:21.003882+0000","last_clean":"2024-08-15T17:59:21.003882+0000","last_became_active":"2024-08-15T17:59:21.000651+0000","last_became_peered":"2024-08-15T17:59:21.000651+0000","last_unstale":"2024-08-15T17:59:21.003882+0000","last_undegraded":"2024-08-15T17:59:21.003882+0000","last_fullsized":"2024-08-15T17:59:21.003882+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:26:32.993579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.018217+0000","last_change":"2024-08-15T17:59:20.016257+0000","last_active":"2024-08-15T17:59:21.018217+0000","last_peered":"2024-08-15T17:59:21.018217+0000","last_clean":"2024-08-15T17:59:21.018217+0000","last_became_active":"2024-08-15T17:59:20.015878+0000","last_became_peered":"2024-08-15T17:59:20.015878+0000","last_unstale":"2024-08-15T17:59:21.018217+0000","last_undegraded":"2024-08-15T17:59:21.018217+0000","last_fullsized":"2024-08-15T17:59:21.018217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:07:52.921234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016804+0000","last_change":"2024-08-15T17:59:21.014608+0000","last_active":"2024-08-15T17:59:21.016804+0000","last_peered":"2024-08-15T17:59:21.016804+0000","last_clean":"2024-08-15T17:59:21.016804+0000","last_became_active":"2024-08-15T17:59:21.010794+0000","last_became_peered":"2024-08-15T17:59:21.010794+0000","last_unstale":"2024-08-15T17:59:21.016804+0000","last_undegraded":"2024-08-15T17:59:21.016804+0000","last_fullsized":"2024-08-15T17:59:21.016804+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:09:21.391162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.017043+0000","last_change":"2024-08-15T17:59:20.001118+0000","last_active":"2024-08-15T17:59:21.017043+0000","last_peered":"2024-08-15T17:59:21.017043+0000","last_clean":"2024-08-15T17:59:21.017043+0000","last_became_active":"2024-08-15T17:59:20.000811+0000","last_became_peered":"2024-08-15T17:59:20.000811+0000","last_unstale":"2024-08-15T17:59:21.017043+0000","last_undegraded":"2024-08-15T17:59:21.017043+0000","last_fullsized":"2024-08-15T17:59:21.017043+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:11.776526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.000931+0000","last_change":"2024-08-15T17:59:21.000931+0000","last_active":"2024-08-15T17:59:21.000931+0000","last_peered":"2024-08-15T17:59:21.000931+0000","last_clean":"2024-08-15T17:59:21.000931+0000","last_became_active":"2024-08-15T17:59:21.000655+0000","last_became_peered":"2024-08-15T17:59:21.000655+0000","last_unstale":"2024-08-15T17:59:21.000931+0000","last_undegraded":"2024-08-15T17:59:21.000931+0000","last_fullsized":"2024-08-15T17:59:21.000931+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:05:35.938370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.987678+0000","last_change":"2024-08-15T17:59:19.999710+0000","last_active":"2024-08-15T17:59:20.987678+0000","last_peered":"2024-08-15T17:59:20.987678+0000","last_clean":"2024-08-15T17:59:20.987678+0000","last_became_active":"2024-08-15T17:59:19.999414+0000","last_became_peered":"2024-08-15T17:59:19.999414+0000","last_unstale":"2024-08-15T17:59:20.987678+0000","last_undegraded":"2024-08-15T17:59:20.987678+0000","last_fullsized":"2024-08-15T17:59:20.987678+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:34:14.250494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.005784+0000","last_change":"2024-08-15T17:59:21.005784+0000","last_active":"2024-08-15T17:59:21.005784+0000","last_peered":"2024-08-15T17:59:21.005784+0000","last_clean":"2024-08-15T17:59:21.005784+0000","last_became_active":"2024-08-15T17:59:21.005502+0000","last_became_peered":"2024-08-15T17:59:21.005502+0000","last_unstale":"2024-08-15T17:59:21.005784+0000","last_undegraded":"2024-08-15T17:59:21.005784+0000","last_fullsized":"2024-08-15T17:59:21.005784+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:52:24.811634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.986692+0000","last_change":"2024-08-15T17:59:20.007939+0000","last_active":"2024-08-15T17:59:20.986692+0000","last_peered":"2024-08-15T17:59:20.986692+0000","last_clean":"2024-08-15T17:59:20.986692+0000","last_became_active":"2024-08-15T17:59:20.007589+0000","last_became_peered":"2024-08-15T17:59:20.007589+0000","last_unstale":"2024-08-15T17:59:20.986692+0000","last_undegraded":"2024-08-15T17:59:20.986692+0000","last_fullsized":"2024-08-15T17:59:20.986692+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:06:25.333482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.997432+0000","last_change":"2024-08-15T17:59:20.997432+0000","last_active":"2024-08-15T17:59:20.997432+0000","last_peered":"2024-08-15T17:59:20.997432+0000","last_clean":"2024-08-15T17:59:20.997432+0000","last_became_active":"2024-08-15T17:59:20.997121+0000","last_became_peered":"2024-08-15T17:59:20.997121+0000","last_unstale":"2024-08-15T17:59:20.997432+0000","last_undegraded":"2024-08-15T17:59:20.997432+0000","last_fullsized":"2024-08-15T17:59:20.997432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:38:59.592614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.016202+0000","last_change":"2024-08-15T17:59:20.010821+0000","last_active":"2024-08-15T17:59:20.016202+0000","last_peered":"2024-08-15T17:59:20.016202+0000","last_clean":"2024-08-15T17:59:20.016202+0000","last_became_active":"2024-08-15T17:59:20.010520+0000","last_became_peered":"2024-08-15T17:59:20.010520+0000","last_unstale":"2024-08-15T17:59:20.016202+0000","last_undegraded":"2024-08-15T17:59:20.016202+0000","last_fullsized":"2024-08-15T17:59:20.016202+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:20:00.781537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016334+0000","last_change":"2024-08-15T17:59:21.012454+0000","last_active":"2024-08-15T17:59:21.016334+0000","last_peered":"2024-08-15T17:59:21.016334+0000","last_clean":"2024-08-15T17:59:21.016334+0000","last_became_active":"2024-08-15T17:59:21.011904+0000","last_became_peered":"2024-08-15T17:59:21.011904+0000","last_unstale":"2024-08-15T17:59:21.016334+0000","last_undegraded":"2024-08-15T17:59:21.016334+0000","last_fullsized":"2024-08-15T17:59:21.016334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:00:04.873888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:19.993710+0000","last_change":"2024-08-15T17:59:19.988445+0000","last_active":"2024-08-15T17:59:19.993710+0000","last_peered":"2024-08-15T17:59:19.993710+0000","last_clean":"2024-08-15T17:59:19.993710+0000","last_became_active":"2024-08-15T17:59:19.987956+0000","last_became_peered":"2024-08-15T17:59:19.987956+0000","last_unstale":"2024-08-15T17:59:19.993710+0000","last_undegraded":"2024-08-15T17:59:19.993710+0000","last_fullsized":"2024-08-15T17:59:19.993710+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:21:50.783043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003008+0000","last_change":"2024-08-15T17:59:21.003008+0000","last_active":"2024-08-15T17:59:21.003008+0000","last_peered":"2024-08-15T17:59:21.003008+0000","last_clean":"2024-08-15T17:59:21.003008+0000","last_became_active":"2024-08-15T17:59:21.002745+0000","last_became_peered":"2024-08-15T17:59:21.002745+0000","last_unstale":"2024-08-15T17:59:21.003008+0000","last_undegraded":"2024-08-15T17:59:21.003008+0000","last_fullsized":"2024-08-15T17:59:21.003008+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:47:35.841722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.015583+0000","last_change":"2024-08-15T17:59:20.008416+0000","last_active":"2024-08-15T17:59:20.015583+0000","last_peered":"2024-08-15T17:59:20.015583+0000","last_clean":"2024-08-15T17:59:20.015583+0000","last_became_active":"2024-08-15T17:59:20.008139+0000","last_became_peered":"2024-08-15T17:59:20.008139+0000","last_unstale":"2024-08-15T17:59:20.015583+0000","last_undegraded":"2024-08-15T17:59:20.015583+0000","last_fullsized":"2024-08-15T17:59:20.015583+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:31:56.124147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.005761+0000","last_change":"2024-08-15T17:59:21.005761+0000","last_active":"2024-08-15T17:59:21.005761+0000","last_peered":"2024-08-15T17:59:21.005761+0000","last_clean":"2024-08-15T17:59:21.005761+0000","last_became_active":"2024-08-15T17:59:21.005094+0000","last_became_peered":"2024-08-15T17:59:21.005094+0000","last_unstale":"2024-08-15T17:59:21.005761+0000","last_undegraded":"2024-08-15T17:59:21.005761+0000","last_fullsized":"2024-08-15T17:59:21.005761+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:02.921095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.986464+0000","last_change":"2024-08-15T17:59:20.009580+0000","last_active":"2024-08-15T17:59:20.986464+0000","last_peered":"2024-08-15T17:59:20.986464+0000","last_clean":"2024-08-15T17:59:20.986464+0000","last_became_active":"2024-08-15T17:59:20.009285+0000","last_became_peered":"2024-08-15T17:59:20.009285+0000","last_unstale":"2024-08-15T17:59:20.986464+0000","last_undegraded":"2024-08-15T17:59:20.986464+0000","last_fullsized":"2024-08-15T17:59:20.986464+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:34:42.148689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002880+0000","last_change":"2024-08-15T17:59:21.002880+0000","last_active":"2024-08-15T17:59:21.002880+0000","last_peered":"2024-08-15T17:59:21.002880+0000","last_clean":"2024-08-15T17:59:21.002880+0000","last_became_active":"2024-08-15T17:59:21.002506+0000","last_became_peered":"2024-08-15T17:59:21.002506+0000","last_unstale":"2024-08-15T17:59:21.002880+0000","last_undegraded":"2024-08-15T17:59:21.002880+0000","last_fullsized":"2024-08-15T17:59:21.002880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:14:27.472146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016852+0000","last_change":"2024-08-15T17:59:19.999985+0000","last_active":"2024-08-15T17:59:21.016852+0000","last_peered":"2024-08-15T17:59:21.016852+0000","last_clean":"2024-08-15T17:59:21.016852+0000","last_became_active":"2024-08-15T17:59:19.999818+0000","last_became_peered":"2024-08-15T17:59:19.999818+0000","last_unstale":"2024-08-15T17:59:21.016852+0000","last_undegraded":"2024-08-15T17:59:21.016852+0000","last_fullsized":"2024-08-15T17:59:21.016852+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:16:36.508757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004125+0000","last_change":"2024-08-15T17:59:21.004125+0000","last_active":"2024-08-15T17:59:21.004125+0000","last_peered":"2024-08-15T17:59:21.004125+0000","last_clean":"2024-08-15T17:59:21.004125+0000","last_became_active":"2024-08-15T17:59:21.003865+0000","last_became_peered":"2024-08-15T17:59:21.003865+0000","last_unstale":"2024-08-15T17:59:21.004125+0000","last_undegraded":"2024-08-15T17:59:21.004125+0000","last_fullsized":"2024-08-15T17:59:21.004125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:14:27.472146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.019686+0000","last_change":"2024-08-15T17:59:20.016970+0000","last_active":"2024-08-15T17:59:21.019686+0000","last_peered":"2024-08-15T17:59:21.019686+0000","last_clean":"2024-08-15T17:59:21.019686+0000","last_became_active":"2024-08-15T17:59:20.016709+0000","last_became_peered":"2024-08-15T17:59:20.016709+0000","last_unstale":"2024-08-15T17:59:21.019686+0000","last_undegraded":"2024-08-15T17:59:21.019686+0000","last_fullsized":"2024-08-15T17:59:21.019686+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:18:33.377985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016588+0000","last_change":"2024-08-15T17:59:21.014942+0000","last_active":"2024-08-15T17:59:21.016588+0000","last_peered":"2024-08-15T17:59:21.016588+0000","last_clean":"2024-08-15T17:59:21.016588+0000","last_became_active":"2024-08-15T17:59:21.012197+0000","last_became_peered":"2024-08-15T17:59:21.012197+0000","last_unstale":"2024-08-15T17:59:21.016588+0000","last_undegraded":"2024-08-15T17:59:21.016588+0000","last_fullsized":"2024-08-15T17:59:21.016588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:58:36.501838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016119+0000","last_change":"2024-08-15T17:59:19.990509+0000","last_active":"2024-08-15T17:59:21.016119+0000","last_peered":"2024-08-15T17:59:21.016119+0000","last_clean":"2024-08-15T17:59:21.016119+0000","last_became_active":"2024-08-15T17:59:19.990207+0000","last_became_peered":"2024-08-15T17:59:19.990207+0000","last_unstale":"2024-08-15T17:59:21.016119+0000","last_undegraded":"2024-08-15T17:59:21.016119+0000","last_fullsized":"2024-08-15T17:59:21.016119+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:23:47.220851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.009311+0000","last_change":"2024-08-15T17:59:21.009311+0000","last_active":"2024-08-15T17:59:21.009311+0000","last_peered":"2024-08-15T17:59:21.009311+0000","last_clean":"2024-08-15T17:59:21.009311+0000","last_became_active":"2024-08-15T17:59:21.008998+0000","last_became_peered":"2024-08-15T17:59:21.008998+0000","last_unstale":"2024-08-15T17:59:21.009311+0000","last_undegraded":"2024-08-15T17:59:21.009311+0000","last_fullsized":"2024-08-15T17:59:21.009311+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:33:04.015051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002069+0000","last_change":"2024-08-15T17:59:20.006965+0000","last_active":"2024-08-15T17:59:21.002069+0000","last_peered":"2024-08-15T17:59:21.002069+0000","last_clean":"2024-08-15T17:59:21.002069+0000","last_became_active":"2024-08-15T17:59:20.006607+0000","last_became_peered":"2024-08-15T17:59:20.006607+0000","last_unstale":"2024-08-15T17:59:21.002069+0000","last_undegraded":"2024-08-15T17:59:21.002069+0000","last_fullsized":"2024-08-15T17:59:21.002069+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:09:00.911809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003202+0000","last_change":"2024-08-15T17:59:21.003202+0000","last_active":"2024-08-15T17:59:21.003202+0000","last_peered":"2024-08-15T17:59:21.003202+0000","last_clean":"2024-08-15T17:59:21.003202+0000","last_became_active":"2024-08-15T17:59:21.002546+0000","last_became_peered":"2024-08-15T17:59:21.002546+0000","last_unstale":"2024-08-15T17:59:21.003202+0000","last_undegraded":"2024-08-15T17:59:21.003202+0000","last_fullsized":"2024-08-15T17:59:21.003202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:32:46.254744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.015535+0000","last_change":"2024-08-15T17:59:20.008440+0000","last_active":"2024-08-15T17:59:20.015535+0000","last_peered":"2024-08-15T17:59:20.015535+0000","last_clean":"2024-08-15T17:59:20.015535+0000","last_became_active":"2024-08-15T17:59:20.008129+0000","last_became_peered":"2024-08-15T17:59:20.008129+0000","last_unstale":"2024-08-15T17:59:20.015535+0000","last_undegraded":"2024-08-15T17:59:20.015535+0000","last_fullsized":"2024-08-15T17:59:20.015535+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:39.995754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.009568+0000","last_change":"2024-08-15T17:59:21.001572+0000","last_active":"2024-08-15T17:59:21.009568+0000","last_peered":"2024-08-15T17:59:21.009568+0000","last_clean":"2024-08-15T17:59:21.009568+0000","last_became_active":"2024-08-15T17:59:21.001291+0000","last_became_peered":"2024-08-15T17:59:21.001291+0000","last_unstale":"2024-08-15T17:59:21.009568+0000","last_undegraded":"2024-08-15T17:59:21.009568+0000","last_fullsized":"2024-08-15T17:59:21.009568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:38:59.592614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.020179+0000","last_change":"2024-08-15T17:59:20.005726+0000","last_active":"2024-08-15T17:59:21.020179+0000","last_peered":"2024-08-15T17:59:21.020179+0000","last_clean":"2024-08-15T17:59:21.020179+0000","last_became_active":"2024-08-15T17:59:20.005441+0000","last_became_peered":"2024-08-15T17:59:20.005441+0000","last_unstale":"2024-08-15T17:59:21.020179+0000","last_undegraded":"2024-08-15T17:59:21.020179+0000","last_fullsized":"2024-08-15T17:59:21.020179+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:35:23.963840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.000227+0000","last_change":"2024-08-15T17:59:21.000227+0000","last_active":"2024-08-15T17:59:21.000227+0000","last_peered":"2024-08-15T17:59:21.000227+0000","last_clean":"2024-08-15T17:59:21.000227+0000","last_became_active":"2024-08-15T17:59:20.999745+0000","last_became_peered":"2024-08-15T17:59:20.999745+0000","last_unstale":"2024-08-15T17:59:21.000227+0000","last_undegraded":"2024-08-15T17:59:21.000227+0000","last_fullsized":"2024-08-15T17:59:21.000227+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:31:01.411361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.018763+0000","last_change":"2024-08-15T17:59:20.016865+0000","last_active":"2024-08-15T17:59:21.018763+0000","last_peered":"2024-08-15T17:59:21.018763+0000","last_clean":"2024-08-15T17:59:21.018763+0000","last_became_active":"2024-08-15T17:59:20.016552+0000","last_became_peered":"2024-08-15T17:59:20.016552+0000","last_unstale":"2024-08-15T17:59:21.018763+0000","last_undegraded":"2024-08-15T17:59:21.018763+0000","last_fullsized":"2024-08-15T17:59:21.018763+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:06:37.220646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003065+0000","last_change":"2024-08-15T17:59:21.003065+0000","last_active":"2024-08-15T17:59:21.003065+0000","last_peered":"2024-08-15T17:59:21.003065+0000","last_clean":"2024-08-15T17:59:21.003065+0000","last_became_active":"2024-08-15T17:59:21.002635+0000","last_became_peered":"2024-08-15T17:59:21.002635+0000","last_unstale":"2024-08-15T17:59:21.003065+0000","last_undegraded":"2024-08-15T17:59:21.003065+0000","last_fullsized":"2024-08-15T17:59:21.003065+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:33:36.843048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016232+0000","last_change":"2024-08-15T17:59:19.990857+0000","last_active":"2024-08-15T17:59:21.016232+0000","last_peered":"2024-08-15T17:59:21.016232+0000","last_clean":"2024-08-15T17:59:21.016232+0000","last_became_active":"2024-08-15T17:59:19.990561+0000","last_became_peered":"2024-08-15T17:59:19.990561+0000","last_unstale":"2024-08-15T17:59:21.016232+0000","last_undegraded":"2024-08-15T17:59:21.016232+0000","last_fullsized":"2024-08-15T17:59:21.016232+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:52:21.522584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016638+0000","last_change":"2024-08-15T17:59:21.008754+0000","last_active":"2024-08-15T17:59:21.016638+0000","last_peered":"2024-08-15T17:59:21.016638+0000","last_clean":"2024-08-15T17:59:21.016638+0000","last_became_active":"2024-08-15T17:59:21.008461+0000","last_became_peered":"2024-08-15T17:59:21.008461+0000","last_unstale":"2024-08-15T17:59:21.016638+0000","last_undegraded":"2024-08-15T17:59:21.016638+0000","last_fullsized":"2024-08-15T17:59:21.016638+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:26:53.696075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.986132+0000","last_change":"2024-08-15T17:59:19.992818+0000","last_active":"2024-08-15T17:59:20.986132+0000","last_peered":"2024-08-15T17:59:20.986132+0000","last_clean":"2024-08-15T17:59:20.986132+0000","last_became_active":"2024-08-15T17:59:19.992480+0000","last_became_peered":"2024-08-15T17:59:19.992480+0000","last_unstale":"2024-08-15T17:59:20.986132+0000","last_undegraded":"2024-08-15T17:59:20.986132+0000","last_fullsized":"2024-08-15T17:59:20.986132+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:20:01.760150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.997507+0000","last_change":"2024-08-15T17:59:20.997507+0000","last_active":"2024-08-15T17:59:20.997507+0000","last_peered":"2024-08-15T17:59:20.997507+0000","last_clean":"2024-08-15T17:59:20.997507+0000","last_became_active":"2024-08-15T17:59:20.997178+0000","last_became_peered":"2024-08-15T17:59:20.997178+0000","last_unstale":"2024-08-15T17:59:20.997507+0000","last_undegraded":"2024-08-15T17:59:20.997507+0000","last_fullsized":"2024-08-15T17:59:20.997507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:20:02.769792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.007273+0000","last_change":"2024-08-15T17:59:19.999638+0000","last_active":"2024-08-15T17:59:20.007273+0000","last_peered":"2024-08-15T17:59:20.007273+0000","last_clean":"2024-08-15T17:59:20.007273+0000","last_became_active":"2024-08-15T17:59:19.999395+0000","last_became_peered":"2024-08-15T17:59:19.999395+0000","last_unstale":"2024-08-15T17:59:20.007273+0000","last_undegraded":"2024-08-15T17:59:20.007273+0000","last_fullsized":"2024-08-15T17:59:20.007273+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:04:43.875545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.996319+0000","last_change":"2024-08-15T17:59:20.996319+0000","last_active":"2024-08-15T17:59:20.996319+0000","last_peered":"2024-08-15T17:59:20.996319+0000","last_clean":"2024-08-15T17:59:20.996319+0000","last_became_active":"2024-08-15T17:59:20.995697+0000","last_became_peered":"2024-08-15T17:59:20.995697+0000","last_unstale":"2024-08-15T17:59:20.996319+0000","last_undegraded":"2024-08-15T17:59:20.996319+0000","last_fullsized":"2024-08-15T17:59:20.996319+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:52:22.532226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.017965+0000","last_change":"2024-08-15T17:59:20.011231+0000","last_active":"2024-08-15T17:59:21.017965+0000","last_peered":"2024-08-15T17:59:21.017965+0000","last_clean":"2024-08-15T17:59:21.017965+0000","last_became_active":"2024-08-15T17:59:20.010939+0000","last_became_peered":"2024-08-15T17:59:20.010939+0000","last_unstale":"2024-08-15T17:59:21.017965+0000","last_undegraded":"2024-08-15T17:59:21.017965+0000","last_fullsized":"2024-08-15T17:59:21.017965+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:59:42.964222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002464+0000","last_change":"2024-08-15T17:59:21.002464+0000","last_active":"2024-08-15T17:59:21.002464+0000","last_peered":"2024-08-15T17:59:21.002464+0000","last_clean":"2024-08-15T17:59:21.002464+0000","last_became_active":"2024-08-15T17:59:21.002069+0000","last_became_peered":"2024-08-15T17:59:21.002069+0000","last_unstale":"2024-08-15T17:59:21.002464+0000","last_undegraded":"2024-08-15T17:59:21.002464+0000","last_fullsized":"2024-08-15T17:59:21.002464+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:30.986582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.994584+0000","last_change":"2024-08-15T17:59:19.999099+0000","last_active":"2024-08-15T17:59:20.994584+0000","last_peered":"2024-08-15T17:59:20.994584+0000","last_clean":"2024-08-15T17:59:20.994584+0000","last_became_active":"2024-08-15T17:59:19.998790+0000","last_became_peered":"2024-08-15T17:59:19.998790+0000","last_unstale":"2024-08-15T17:59:20.994584+0000","last_undegraded":"2024-08-15T17:59:20.994584+0000","last_fullsized":"2024-08-15T17:59:20.994584+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:52:04.693417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002740+0000","last_change":"2024-08-15T17:59:21.002740+0000","last_active":"2024-08-15T17:59:21.002740+0000","last_peered":"2024-08-15T17:59:21.002740+0000","last_clean":"2024-08-15T17:59:21.002740+0000","last_became_active":"2024-08-15T17:59:21.002240+0000","last_became_peered":"2024-08-15T17:59:21.002240+0000","last_unstale":"2024-08-15T17:59:21.002740+0000","last_undegraded":"2024-08-15T17:59:21.002740+0000","last_fullsized":"2024-08-15T17:59:21.002740+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:06:26.343124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.001827+0000","last_change":"2024-08-15T17:59:19.997704+0000","last_active":"2024-08-15T17:59:20.001827+0000","last_peered":"2024-08-15T17:59:20.001827+0000","last_clean":"2024-08-15T17:59:20.001827+0000","last_became_active":"2024-08-15T17:59:19.997413+0000","last_became_peered":"2024-08-15T17:59:19.997413+0000","last_unstale":"2024-08-15T17:59:20.001827+0000","last_undegraded":"2024-08-15T17:59:20.001827+0000","last_fullsized":"2024-08-15T17:59:20.001827+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:38:21.644793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.010619+0000","last_change":"2024-08-15T17:59:21.010619+0000","last_active":"2024-08-15T17:59:21.010619+0000","last_peered":"2024-08-15T17:59:21.010619+0000","last_clean":"2024-08-15T17:59:21.010619+0000","last_became_active":"2024-08-15T17:59:21.010311+0000","last_became_peered":"2024-08-15T17:59:21.010311+0000","last_unstale":"2024-08-15T17:59:21.010619+0000","last_undegraded":"2024-08-15T17:59:21.010619+0000","last_fullsized":"2024-08-15T17:59:21.010619+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:26:33.249200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.004713+0000","last_change":"2024-08-15T17:59:20.000815+0000","last_active":"2024-08-15T17:59:20.004713+0000","last_peered":"2024-08-15T17:59:20.004713+0000","last_clean":"2024-08-15T17:59:20.004713+0000","last_became_active":"2024-08-15T17:59:20.000540+0000","last_became_peered":"2024-08-15T17:59:20.000540+0000","last_unstale":"2024-08-15T17:59:20.004713+0000","last_undegraded":"2024-08-15T17:59:20.004713+0000","last_fullsized":"2024-08-15T17:59:20.004713+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:00:59.526752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.007597+0000","last_change":"2024-08-15T17:59:21.007597+0000","last_active":"2024-08-15T17:59:21.007597+0000","last_peered":"2024-08-15T17:59:21.007597+0000","last_clean":"2024-08-15T17:59:21.007597+0000","last_became_active":"2024-08-15T17:59:21.007314+0000","last_became_peered":"2024-08-15T17:59:21.007314+0000","last_unstale":"2024-08-15T17:59:21.007597+0000","last_undegraded":"2024-08-15T17:59:21.007597+0000","last_fullsized":"2024-08-15T17:59:21.007597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:38:22.654435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.020691+0000","last_change":"2024-08-15T17:59:20.005291+0000","last_active":"2024-08-15T17:59:21.020691+0000","last_peered":"2024-08-15T17:59:21.020691+0000","last_clean":"2024-08-15T17:59:21.020691+0000","last_became_active":"2024-08-15T17:59:20.004900+0000","last_became_peered":"2024-08-15T17:59:20.004900+0000","last_unstale":"2024-08-15T17:59:21.020691+0000","last_undegraded":"2024-08-15T17:59:21.020691+0000","last_fullsized":"2024-08-15T17:59:21.020691+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:43:52.441863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003858+0000","last_change":"2024-08-15T17:59:21.003858+0000","last_active":"2024-08-15T17:59:21.003858+0000","last_peered":"2024-08-15T17:59:21.003858+0000","last_clean":"2024-08-15T17:59:21.003858+0000","last_became_active":"2024-08-15T17:59:21.003576+0000","last_became_peered":"2024-08-15T17:59:21.003576+0000","last_unstale":"2024-08-15T17:59:21.003858+0000","last_undegraded":"2024-08-15T17:59:21.003858+0000","last_fullsized":"2024-08-15T17:59:21.003858+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:36:38.729269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.986536+0000","last_change":"2024-08-15T17:59:20.011484+0000","last_active":"2024-08-15T17:59:20.986536+0000","last_peered":"2024-08-15T17:59:20.986536+0000","last_clean":"2024-08-15T17:59:20.986536+0000","last_became_active":"2024-08-15T17:59:20.011164+0000","last_became_peered":"2024-08-15T17:59:20.011164+0000","last_unstale":"2024-08-15T17:59:20.986536+0000","last_undegraded":"2024-08-15T17:59:20.986536+0000","last_fullsized":"2024-08-15T17:59:20.986536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:10:46.666183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.009384+0000","last_change":"2024-08-15T17:59:21.003804+0000","last_active":"2024-08-15T17:59:21.009384+0000","last_peered":"2024-08-15T17:59:21.009384+0000","last_clean":"2024-08-15T17:59:21.009384+0000","last_became_active":"2024-08-15T17:59:21.003448+0000","last_became_peered":"2024-08-15T17:59:21.003448+0000","last_unstale":"2024-08-15T17:59:21.009384+0000","last_undegraded":"2024-08-15T17:59:21.009384+0000","last_fullsized":"2024-08-15T17:59:21.009384+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:07:53.930876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002553+0000","last_change":"2024-08-15T17:59:20.010613+0000","last_active":"2024-08-15T17:59:21.002553+0000","last_peered":"2024-08-15T17:59:21.002553+0000","last_clean":"2024-08-15T17:59:21.002553+0000","last_became_active":"2024-08-15T17:59:20.010288+0000","last_became_peered":"2024-08-15T17:59:20.010288+0000","last_unstale":"2024-08-15T17:59:21.002553+0000","last_undegraded":"2024-08-15T17:59:21.002553+0000","last_fullsized":"2024-08-15T17:59:21.002553+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:05:39.068362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.013695+0000","last_change":"2024-08-15T17:59:21.007020+0000","last_active":"2024-08-15T17:59:21.013695+0000","last_peered":"2024-08-15T17:59:21.013695+0000","last_clean":"2024-08-15T17:59:21.013695+0000","last_became_active":"2024-08-15T17:59:21.006713+0000","last_became_peered":"2024-08-15T17:59:21.006713+0000","last_unstale":"2024-08-15T17:59:21.013695+0000","last_undegraded":"2024-08-15T17:59:21.013695+0000","last_fullsized":"2024-08-15T17:59:21.013695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:31:01.411361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.985601+0000","last_change":"2024-08-15T17:59:20.004178+0000","last_active":"2024-08-15T17:59:20.985601+0000","last_peered":"2024-08-15T17:59:20.985601+0000","last_clean":"2024-08-15T17:59:20.985601+0000","last_became_active":"2024-08-15T17:59:20.003882+0000","last_became_peered":"2024-08-15T17:59:20.003882+0000","last_unstale":"2024-08-15T17:59:20.985601+0000","last_undegraded":"2024-08-15T17:59:20.985601+0000","last_fullsized":"2024-08-15T17:59:20.985601+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:14:30.395953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.015866+0000","last_change":"2024-08-15T17:59:21.009520+0000","last_active":"2024-08-15T17:59:21.015866+0000","last_peered":"2024-08-15T17:59:21.015866+0000","last_clean":"2024-08-15T17:59:21.015866+0000","last_became_active":"2024-08-15T17:59:21.009012+0000","last_became_peered":"2024-08-15T17:59:21.009012+0000","last_unstale":"2024-08-15T17:59:21.015866+0000","last_undegraded":"2024-08-15T17:59:21.015866+0000","last_fullsized":"2024-08-15T17:59:21.015866+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:31:57.133789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002204+0000","last_change":"2024-08-15T17:59:20.005853+0000","last_active":"2024-08-15T17:59:21.002204+0000","last_peered":"2024-08-15T17:59:21.002204+0000","last_clean":"2024-08-15T17:59:21.002204+0000","last_became_active":"2024-08-15T17:59:20.005555+0000","last_became_peered":"2024-08-15T17:59:20.005555+0000","last_unstale":"2024-08-15T17:59:21.002204+0000","last_undegraded":"2024-08-15T17:59:21.002204+0000","last_fullsized":"2024-08-15T17:59:21.002204+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:52:04.693417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003843+0000","last_change":"2024-08-15T17:59:21.003843+0000","last_active":"2024-08-15T17:59:21.003843+0000","last_peered":"2024-08-15T17:59:21.003843+0000","last_clean":"2024-08-15T17:59:21.003843+0000","last_became_active":"2024-08-15T17:59:21.003533+0000","last_became_peered":"2024-08-15T17:59:21.003533+0000","last_unstale":"2024-08-15T17:59:21.003843+0000","last_undegraded":"2024-08-15T17:59:21.003843+0000","last_fullsized":"2024-08-15T17:59:21.003843+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:47:35.841722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:19.998991+0000","last_change":"2024-08-15T17:59:19.992086+0000","last_active":"2024-08-15T17:59:19.998991+0000","last_peered":"2024-08-15T17:59:19.998991+0000","last_clean":"2024-08-15T17:59:19.998991+0000","last_became_active":"2024-08-15T17:59:19.991727+0000","last_became_peered":"2024-08-15T17:59:19.991727+0000","last_unstale":"2024-08-15T17:59:19.998991+0000","last_undegraded":"2024-08-15T17:59:19.998991+0000","last_fullsized":"2024-08-15T17:59:19.998991+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:18:57.157654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004762+0000","last_change":"2024-08-15T17:59:21.004762+0000","last_active":"2024-08-15T17:59:21.004762+0000","last_peered":"2024-08-15T17:59:21.004762+0000","last_clean":"2024-08-15T17:59:21.004762+0000","last_became_active":"2024-08-15T17:59:21.003777+0000","last_became_peered":"2024-08-15T17:59:21.003777+0000","last_unstale":"2024-08-15T17:59:21.004762+0000","last_undegraded":"2024-08-15T17:59:21.004762+0000","last_fullsized":"2024-08-15T17:59:21.004762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:30:10.217672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.015698+0000","last_change":"2024-08-15T17:59:20.008771+0000","last_active":"2024-08-15T17:59:20.015698+0000","last_peered":"2024-08-15T17:59:20.015698+0000","last_clean":"2024-08-15T17:59:20.015698+0000","last_became_active":"2024-08-15T17:59:20.008494+0000","last_became_peered":"2024-08-15T17:59:20.008494+0000","last_unstale":"2024-08-15T17:59:20.015698+0000","last_undegraded":"2024-08-15T17:59:20.015698+0000","last_fullsized":"2024-08-15T17:59:20.015698+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:15:04.414906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.008525+0000","last_change":"2024-08-15T17:59:21.008525+0000","last_active":"2024-08-15T17:59:21.008525+0000","last_peered":"2024-08-15T17:59:21.008525+0000","last_clean":"2024-08-15T17:59:21.008525+0000","last_became_active":"2024-08-15T17:59:21.008253+0000","last_became_peered":"2024-08-15T17:59:21.008253+0000","last_unstale":"2024-08-15T17:59:21.008525+0000","last_undegraded":"2024-08-15T17:59:21.008525+0000","last_fullsized":"2024-08-15T17:59:21.008525+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:58:21.676184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.994165+0000","last_change":"2024-08-15T17:59:19.993347+0000","last_active":"2024-08-15T17:59:20.994165+0000","last_peered":"2024-08-15T17:59:20.994165+0000","last_clean":"2024-08-15T17:59:20.994165+0000","last_became_active":"2024-08-15T17:59:19.993017+0000","last_became_peered":"2024-08-15T17:59:19.993017+0000","last_unstale":"2024-08-15T17:59:20.994165+0000","last_undegraded":"2024-08-15T17:59:20.994165+0000","last_fullsized":"2024-08-15T17:59:20.994165+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:29.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:19.998192+0000","last_change":"2024-08-15T17:58:24.394014+0000","last_active":"2024-08-15T17:59:19.998192+0000","last_peered":"2024-08-15T17:59:19.998192+0000","last_clean":"2024-08-15T17:59:19.998192+0000","last_became_active":"2024-08-15T17:58:24.391735+0000","last_became_peered":"2024-08-15T17:58:24.391735+0000","last_unstale":"2024-08-15T17:59:19.998192+0000","last_undegraded":"2024-08-15T17:59:19.998192+0000","last_fullsized":"2024-08-15T17:59:19.998192+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_clean_scrub_stamp":"2024-08-15T17:58:22.996335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:31.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302440,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069400,"statfs":{"total":96636764160,"available":96327065600,"internally_reserved":0,"allocated":937984,"data_stored":720031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297708,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074132,"statfs":{"total":96636764160,"available":96331911168,"internally_reserved":0,"allocated":360448,"data_stored":136465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]}]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":348160,"data_stored":129677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301856,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069984,"statfs":{"total":96636764160,"available":96327663616,"internally_reserved":0,"allocated":348160,"data_stored":130233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":356352,"data_stored":131147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301852,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069988,"statfs":{"total":96636764160,"available":96327667712,"internally_reserved":0,"allocated":344064,"data_stored":129643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":356352,"data_stored":131147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298268,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073572,"statfs":{"total":96636764160,"available":96331337728,"internally_reserved":0,"allocated":933888,"data_stored":719941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:59:25.024 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:59:25.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.129+0000 7fca56093700 1 Processor -- start 2024-08-15T17:59:25.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.129+0000 7fca56093700 1 -- start start 2024-08-15T17:59:25.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fca500b2840 0x7fca501580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fca50158b50 0x7fca50158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca501585f0 con 0x7fca5015a580 2024-08-15T17:59:25.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca5015bc00 con 0x7fca500b2840 2024-08-15T17:59:25.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca5015aff0 con 0x7fca50158b50 2024-08-15T17:59:25.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca47fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fca500b2840 0x7fca501580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca47fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fca500b2840 0x7fca501580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41772/0 (socket says 172.21.15.38:41772) 2024-08-15T17:59:25.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca47fff700 1 -- 172.21.15.38:0/70074194 learned_addr learned my addr 172.21.15.38:0/70074194 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:25.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca4ffff700 1 --2- 172.21.15.38:0/70074194 >> v2:172.21.15.104:3300/0 conn(0x7fca50158b50 0x7fca50158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca47fff700 1 -- 172.21.15.38:0/70074194 >> v2:172.21.15.104:3300/0 conn(0x7fca50158b50 msgr2=0x7fca50158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:25.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca47fff700 1 --2- 172.21.15.38:0/70074194 >> v2:172.21.15.104:3300/0 conn(0x7fca50158b50 0x7fca50158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca47fff700 1 -- 172.21.15.38:0/70074194 >> v1:172.21.15.104:6789/0 conn(0x7fca5015a580 legacy=0x7fca500b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca47fff700 1 -- 172.21.15.38:0/70074194 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5015b2d0 con 0x7fca500b2840 2024-08-15T17:59:25.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca47fff700 1 --2- 172.21.15.38:0/70074194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca500b2840 0x7fca501580b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca5006d8f0 tx=0x7fca38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69162fe63aee3e85 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:25.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca4effd700 1 -- 172.21.15.38:0/70074194 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38009e70 con 0x7fca500b2840 2024-08-15T17:59:25.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca4effd700 1 -- 172.21.15.38:0/70074194 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3800b860 con 0x7fca500b2840 2024-08-15T17:59:25.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca4effd700 1 -- 172.21.15.38:0/70074194 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3800bc00 con 0x7fca500b2840 2024-08-15T17:59:25.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 -- 172.21.15.38:0/70074194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca500b2840 msgr2=0x7fca501580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 --2- 172.21.15.38:0/70074194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca500b2840 0x7fca501580b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca5006d8f0 tx=0x7fca38009250 comp rx=0 tx=0).stop 2024-08-15T17:59:25.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 -- 172.21.15.38:0/70074194 shutdown_connections 2024-08-15T17:59:25.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 --2- 172.21.15.38:0/70074194 >> v2:172.21.15.104:3300/0 conn(0x7fca50158b50 0x7fca50158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 --2- 172.21.15.38:0/70074194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca500b2840 0x7fca501580b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 -- 172.21.15.38:0/70074194 >> 172.21.15.38:0/70074194 conn(0x7fca500a9db0 msgr2=0x7fca500ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:25.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 -- 172.21.15.38:0/70074194 shutdown_connections 2024-08-15T17:59:25.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.133+0000 7fca56093700 1 -- 172.21.15.38:0/70074194 wait complete. 2024-08-15T17:59:25.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca56093700 1 Processor -- start 2024-08-15T17:59:25.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca56093700 1 -- start start 2024-08-15T17:59:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca56093700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca500b2840 0x7fca50155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca56093700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fca50158b50 0x7fca500c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca56093700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5015a580 0x7fca500aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca56093700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca500c20d0 con 0x7fca500b2840 2024-08-15T17:59:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca56093700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca50169780 con 0x7fca5015a580 2024-08-15T17:59:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca56093700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca500c2c00 con 0x7fca50158b50 2024-08-15T17:59:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca4f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca500b2840 0x7fca50155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca4f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca500b2840 0x7fca50155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41784/0 (socket says 172.21.15.38:41784) 2024-08-15T17:59:25.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca4f7fe700 1 -- 172.21.15.38:0/3057408167 learned_addr learned my addr 172.21.15.38:0/3057408167 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:25.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca47fff700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fca50158b50 0x7fca500c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca4ffff700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5015a580 0x7fca500aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca4f7fe700 1 -- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fca50158b50 msgr2=0x7fca500c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca4f7fe700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fca50158b50 0x7fca500c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca4f7fe700 1 -- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5015a580 msgr2=0x7fca500aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca4f7fe700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5015a580 0x7fca500aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca4f7fe700 1 -- 172.21.15.38:0/3057408167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380094f0 con 0x7fca500b2840 2024-08-15T17:59:25.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca4ffff700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5015a580 0x7fca500aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:25.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca4f7fe700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca500b2840 0x7fca50155d20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fca4000c510 tx=0x7fca4000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:25.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.137+0000 7fca477fe700 1 -- 172.21.15.38:0/3057408167 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40011590 con 0x7fca500b2840 2024-08-15T17:59:25.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.141+0000 7fca477fe700 1 -- 172.21.15.38:0/3057408167 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca40011710 con 0x7fca500b2840 2024-08-15T17:59:25.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.141+0000 7fca56093700 1 -- 172.21.15.38:0/3057408167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca500c2ee0 con 0x7fca500b2840 2024-08-15T17:59:25.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.141+0000 7fca477fe700 1 -- 172.21.15.38:0/3057408167 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40011ad0 con 0x7fca500b2840 2024-08-15T17:59:25.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.141+0000 7fca56093700 1 -- 172.21.15.38:0/3057408167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca501699f0 con 0x7fca500b2840 2024-08-15T17:59:25.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.141+0000 7fca56093700 1 -- 172.21.15.38:0/3057408167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca500645c0 con 0x7fca500b2840 2024-08-15T17:59:25.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.141+0000 7fca477fe700 1 -- 172.21.15.38:0/3057408167 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fca4001b3b0 con 0x7fca500b2840 2024-08-15T17:59:25.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.141+0000 7fca477fe700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fca30060860 0x7fca30062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.145+0000 7fca477fe700 1 -- 172.21.15.38:0/3057408167 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca4007c290 con 0x7fca500b2840 2024-08-15T17:59:25.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.145+0000 7fca47fff700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fca30060860 0x7fca30062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.145+0000 7fca47fff700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fca30060860 0x7fca30062d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca5006d8f0 tx=0x7fca38017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:25.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.149+0000 7fca477fe700 1 -- 172.21.15.38:0/3057408167 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fca4004c760 con 0x7fca500b2840 2024-08-15T17:59:25.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.309+0000 7fca56093700 1 -- 172.21.15.38:0/3057408167 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca501585f0 con 0x7fca30060860 2024-08-15T17:59:25.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.325+0000 7fca477fe700 1 -- 172.21.15.38:0/3057408167 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264608 (secure 0 0 0) 0x7fca501585f0 con 0x7fca30060860 2024-08-15T17:59:25.328 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:25.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 -- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fca30060860 msgr2=0x7fca30062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fca30060860 0x7fca30062d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca5006d8f0 tx=0x7fca38017000 comp rx=0 tx=0).stop 2024-08-15T17:59:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 -- 172.21.15.38:0/3057408167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca500b2840 msgr2=0x7fca50155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca500b2840 0x7fca50155d20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fca4000c510 tx=0x7fca4000c820 comp rx=0 tx=0).stop 2024-08-15T17:59:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 -- 172.21.15.38:0/3057408167 shutdown_connections 2024-08-15T17:59:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fca30060860 0x7fca30062d10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5015a580 0x7fca500aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fca50158b50 0x7fca500c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 --2- 172.21.15.38:0/3057408167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca500b2840 0x7fca50155d20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 -- 172.21.15.38:0/3057408167 >> 172.21.15.38:0/3057408167 conn(0x7fca500a9db0 msgr2=0x7fca500ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 -- 172.21.15.38:0/3057408167 shutdown_connections 2024-08-15T17:59:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.329+0000 7fca56093700 1 -- 172.21.15.38:0/3057408167 wait complete. 2024-08-15T17:59:25.332 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-08-15T17:59:25.344 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-08-15T17:59:24.184619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2399404,"kb_used_data":3892,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752575316,"statfs":{"total":773094113280,"available":770637123584,"internally_reserved":0,"allocated":3985408,"data_stored":2228284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220040"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016143+0000","last_change":"2024-08-15T17:59:21.009226+0000","last_active":"2024-08-15T17:59:21.016143+0000","last_peered":"2024-08-15T17:59:21.016143+0000","last_clean":"2024-08-15T17:59:21.016143+0000","last_became_active":"2024-08-15T17:59:21.007242+0000","last_became_peered":"2024-08-15T17:59:21.007242+0000","last_unstale":"2024-08-15T17:59:21.016143+0000","last_undegraded":"2024-08-15T17:59:21.016143+0000","last_fullsized":"2024-08-15T17:59:21.016143+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:52:47.789349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.006240+0000","last_change":"2024-08-15T17:59:21.006240+0000","last_active":"2024-08-15T17:59:21.006240+0000","last_peered":"2024-08-15T17:59:21.006240+0000","last_clean":"2024-08-15T17:59:21.006240+0000","last_became_active":"2024-08-15T17:59:21.005175+0000","last_became_peered":"2024-08-15T17:59:21.005175+0000","last_unstale":"2024-08-15T17:59:21.006240+0000","last_undegraded":"2024-08-15T17:59:21.006240+0000","last_fullsized":"2024-08-15T17:59:21.006240+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:42:51.216038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004133+0000","last_change":"2024-08-15T17:59:21.004133+0000","last_active":"2024-08-15T17:59:21.004133+0000","last_peered":"2024-08-15T17:59:21.004133+0000","last_clean":"2024-08-15T17:59:21.004133+0000","last_became_active":"2024-08-15T17:59:21.003851+0000","last_became_peered":"2024-08-15T17:59:21.003851+0000","last_unstale":"2024-08-15T17:59:21.004133+0000","last_undegraded":"2024-08-15T17:59:21.004133+0000","last_fullsized":"2024-08-15T17:59:21.004133+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:33:33.720588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016619+0000","last_change":"2024-08-15T17:59:21.008278+0000","last_active":"2024-08-15T17:59:21.016619+0000","last_peered":"2024-08-15T17:59:21.016619+0000","last_clean":"2024-08-15T17:59:21.016619+0000","last_became_active":"2024-08-15T17:59:21.007966+0000","last_became_peered":"2024-08-15T17:59:21.007966+0000","last_unstale":"2024-08-15T17:59:21.016619+0000","last_undegraded":"2024-08-15T17:59:21.016619+0000","last_fullsized":"2024-08-15T17:59:21.016619+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:15:12.056272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004929+0000","last_change":"2024-08-15T17:59:21.004929+0000","last_active":"2024-08-15T17:59:21.004929+0000","last_peered":"2024-08-15T17:59:21.004929+0000","last_clean":"2024-08-15T17:59:21.004929+0000","last_became_active":"2024-08-15T17:59:21.004600+0000","last_became_peered":"2024-08-15T17:59:21.004600+0000","last_unstale":"2024-08-15T17:59:21.004929+0000","last_undegraded":"2024-08-15T17:59:21.004929+0000","last_fullsized":"2024-08-15T17:59:21.004929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:43:53.451504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.007284+0000","last_change":"2024-08-15T17:59:21.007284+0000","last_active":"2024-08-15T17:59:21.007284+0000","last_peered":"2024-08-15T17:59:21.007284+0000","last_clean":"2024-08-15T17:59:21.007284+0000","last_became_active":"2024-08-15T17:59:21.006962+0000","last_became_peered":"2024-08-15T17:59:21.006962+0000","last_unstale":"2024-08-15T17:59:21.007284+0000","last_undegraded":"2024-08-15T17:59:21.007284+0000","last_fullsized":"2024-08-15T17:59:21.007284+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:34:38.302879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.010719+0000","last_change":"2024-08-15T17:59:21.010719+0000","last_active":"2024-08-15T17:59:21.010719+0000","last_peered":"2024-08-15T17:59:21.010719+0000","last_clean":"2024-08-15T17:59:21.010719+0000","last_became_active":"2024-08-15T17:59:21.010425+0000","last_became_peered":"2024-08-15T17:59:21.010425+0000","last_unstale":"2024-08-15T17:59:21.010719+0000","last_undegraded":"2024-08-15T17:59:21.010719+0000","last_fullsized":"2024-08-15T17:59:21.010719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:27:52.825284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.008627+0000","last_change":"2024-08-15T17:59:21.008627+0000","last_active":"2024-08-15T17:59:21.008627+0000","last_peered":"2024-08-15T17:59:21.008627+0000","last_clean":"2024-08-15T17:59:21.008627+0000","last_became_active":"2024-08-15T17:59:21.008366+0000","last_became_peered":"2024-08-15T17:59:21.008366+0000","last_unstale":"2024-08-15T17:59:21.008627+0000","last_undegraded":"2024-08-15T17:59:21.008627+0000","last_fullsized":"2024-08-15T17:59:21.008627+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:17:19.740984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004751+0000","last_change":"2024-08-15T17:59:21.004751+0000","last_active":"2024-08-15T17:59:21.004751+0000","last_peered":"2024-08-15T17:59:21.004751+0000","last_clean":"2024-08-15T17:59:21.004751+0000","last_became_active":"2024-08-15T17:59:21.004215+0000","last_became_peered":"2024-08-15T17:59:21.004215+0000","last_unstale":"2024-08-15T17:59:21.004751+0000","last_undegraded":"2024-08-15T17:59:21.004751+0000","last_fullsized":"2024-08-15T17:59:21.004751+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:09:21.391162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016211+0000","last_change":"2024-08-15T17:59:21.011249+0000","last_active":"2024-08-15T17:59:21.016211+0000","last_peered":"2024-08-15T17:59:21.016211+0000","last_clean":"2024-08-15T17:59:21.016211+0000","last_became_active":"2024-08-15T17:59:21.010935+0000","last_became_peered":"2024-08-15T17:59:21.010935+0000","last_unstale":"2024-08-15T17:59:21.016211+0000","last_undegraded":"2024-08-15T17:59:21.016211+0000","last_fullsized":"2024-08-15T17:59:21.016211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:34:50.258303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004941+0000","last_change":"2024-08-15T17:59:21.004941+0000","last_active":"2024-08-15T17:59:21.004941+0000","last_peered":"2024-08-15T17:59:21.004941+0000","last_clean":"2024-08-15T17:59:21.004941+0000","last_became_active":"2024-08-15T17:59:21.004665+0000","last_became_peered":"2024-08-15T17:59:21.004665+0000","last_unstale":"2024-08-15T17:59:21.004941+0000","last_undegraded":"2024-08-15T17:59:21.004941+0000","last_fullsized":"2024-08-15T17:59:21.004941+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:05:39.941247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.001954+0000","last_change":"2024-08-15T17:59:21.001954+0000","last_active":"2024-08-15T17:59:21.001954+0000","last_peered":"2024-08-15T17:59:21.001954+0000","last_clean":"2024-08-15T17:59:21.001954+0000","last_became_active":"2024-08-15T17:59:21.001503+0000","last_became_peered":"2024-08-15T17:59:21.001503+0000","last_unstale":"2024-08-15T17:59:21.001954+0000","last_undegraded":"2024-08-15T17:59:21.001954+0000","last_fullsized":"2024-08-15T17:59:21.001954+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:20:04.815893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004865+0000","last_change":"2024-08-15T17:59:21.004865+0000","last_active":"2024-08-15T17:59:21.004865+0000","last_peered":"2024-08-15T17:59:21.004865+0000","last_clean":"2024-08-15T17:59:21.004865+0000","last_became_active":"2024-08-15T17:59:21.004538+0000","last_became_peered":"2024-08-15T17:59:21.004538+0000","last_unstale":"2024-08-15T17:59:21.004865+0000","last_undegraded":"2024-08-15T17:59:21.004865+0000","last_fullsized":"2024-08-15T17:59:21.004865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:52:05.703059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.010456+0000","last_change":"2024-08-15T17:59:21.010456+0000","last_active":"2024-08-15T17:59:21.010456+0000","last_peered":"2024-08-15T17:59:21.010456+0000","last_clean":"2024-08-15T17:59:21.010456+0000","last_became_active":"2024-08-15T17:59:21.010081+0000","last_became_peered":"2024-08-15T17:59:21.010081+0000","last_unstale":"2024-08-15T17:59:21.010456+0000","last_undegraded":"2024-08-15T17:59:21.010456+0000","last_fullsized":"2024-08-15T17:59:21.010456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:22:59.034828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.011051+0000","last_change":"2024-08-15T17:59:21.011051+0000","last_active":"2024-08-15T17:59:21.011051+0000","last_peered":"2024-08-15T17:59:21.011051+0000","last_clean":"2024-08-15T17:59:21.011051+0000","last_became_active":"2024-08-15T17:59:21.010785+0000","last_became_peered":"2024-08-15T17:59:21.010785+0000","last_unstale":"2024-08-15T17:59:21.011051+0000","last_undegraded":"2024-08-15T17:59:21.011051+0000","last_fullsized":"2024-08-15T17:59:21.011051+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:41:41.362879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003364+0000","last_change":"2024-08-15T17:59:21.003364+0000","last_active":"2024-08-15T17:59:21.003364+0000","last_peered":"2024-08-15T17:59:21.003364+0000","last_clean":"2024-08-15T17:59:21.003364+0000","last_became_active":"2024-08-15T17:59:21.002797+0000","last_became_peered":"2024-08-15T17:59:21.002797+0000","last_unstale":"2024-08-15T17:59:21.003364+0000","last_undegraded":"2024-08-15T17:59:21.003364+0000","last_fullsized":"2024-08-15T17:59:21.003364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:24:53.455619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.009953+0000","last_change":"2024-08-15T17:59:21.009953+0000","last_active":"2024-08-15T17:59:21.009953+0000","last_peered":"2024-08-15T17:59:21.009953+0000","last_clean":"2024-08-15T17:59:21.009953+0000","last_became_active":"2024-08-15T17:59:21.009698+0000","last_became_peered":"2024-08-15T17:59:21.009698+0000","last_unstale":"2024-08-15T17:59:21.009953+0000","last_undegraded":"2024-08-15T17:59:21.009953+0000","last_fullsized":"2024-08-15T17:59:21.009953+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:58:36.501838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.006436+0000","last_change":"2024-08-15T17:59:21.006436+0000","last_active":"2024-08-15T17:59:21.006436+0000","last_peered":"2024-08-15T17:59:21.006436+0000","last_clean":"2024-08-15T17:59:21.006436+0000","last_became_active":"2024-08-15T17:59:21.005550+0000","last_became_peered":"2024-08-15T17:59:21.005550+0000","last_unstale":"2024-08-15T17:59:21.006436+0000","last_undegraded":"2024-08-15T17:59:21.006436+0000","last_fullsized":"2024-08-15T17:59:21.006436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:23:10.592319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.005895+0000","last_change":"2024-08-15T17:59:21.005895+0000","last_active":"2024-08-15T17:59:21.005895+0000","last_peered":"2024-08-15T17:59:21.005895+0000","last_clean":"2024-08-15T17:59:21.005895+0000","last_became_active":"2024-08-15T17:59:21.005612+0000","last_became_peered":"2024-08-15T17:59:21.005612+0000","last_unstale":"2024-08-15T17:59:21.005895+0000","last_undegraded":"2024-08-15T17:59:21.005895+0000","last_fullsized":"2024-08-15T17:59:21.005895+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:20:04.815893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.008445+0000","last_change":"2024-08-15T17:59:21.008445+0000","last_active":"2024-08-15T17:59:21.008445+0000","last_peered":"2024-08-15T17:59:21.008445+0000","last_clean":"2024-08-15T17:59:21.008445+0000","last_became_active":"2024-08-15T17:59:21.008154+0000","last_became_peered":"2024-08-15T17:59:21.008154+0000","last_unstale":"2024-08-15T17:59:21.008445+0000","last_undegraded":"2024-08-15T17:59:21.008445+0000","last_fullsized":"2024-08-15T17:59:21.008445+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:29:18.099835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.996839+0000","last_change":"2024-08-15T17:59:20.996839+0000","last_active":"2024-08-15T17:59:20.996839+0000","last_peered":"2024-08-15T17:59:20.996839+0000","last_clean":"2024-08-15T17:59:20.996839+0000","last_became_active":"2024-08-15T17:59:20.996480+0000","last_became_peered":"2024-08-15T17:59:20.996480+0000","last_unstale":"2024-08-15T17:59:20.996839+0000","last_undegraded":"2024-08-15T17:59:20.996839+0000","last_fullsized":"2024-08-15T17:59:20.996839+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:14:54.232264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004296+0000","last_change":"2024-08-15T17:59:21.004296+0000","last_active":"2024-08-15T17:59:21.004296+0000","last_peered":"2024-08-15T17:59:21.004296+0000","last_clean":"2024-08-15T17:59:21.004296+0000","last_became_active":"2024-08-15T17:59:21.003472+0000","last_became_peered":"2024-08-15T17:59:21.003472+0000","last_unstale":"2024-08-15T17:59:21.004296+0000","last_undegraded":"2024-08-15T17:59:21.004296+0000","last_fullsized":"2024-08-15T17:59:21.004296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:48:44.709713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004532+0000","last_change":"2024-08-15T17:59:20.996638+0000","last_active":"2024-08-15T17:59:21.004532+0000","last_peered":"2024-08-15T17:59:21.004532+0000","last_clean":"2024-08-15T17:59:21.004532+0000","last_became_active":"2024-08-15T17:59:20.996332+0000","last_became_peered":"2024-08-15T17:59:20.996332+0000","last_unstale":"2024-08-15T17:59:21.004532+0000","last_undegraded":"2024-08-15T17:59:21.004532+0000","last_fullsized":"2024-08-15T17:59:21.004532+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:15:05.424547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.008964+0000","last_change":"2024-08-15T17:59:21.008964+0000","last_active":"2024-08-15T17:59:21.008964+0000","last_peered":"2024-08-15T17:59:21.008964+0000","last_clean":"2024-08-15T17:59:21.008964+0000","last_became_active":"2024-08-15T17:59:21.008709+0000","last_became_peered":"2024-08-15T17:59:21.008709+0000","last_unstale":"2024-08-15T17:59:21.008964+0000","last_undegraded":"2024-08-15T17:59:21.008964+0000","last_fullsized":"2024-08-15T17:59:21.008964+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:46:14.773091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016071+0000","last_change":"2024-08-15T17:59:21.008310+0000","last_active":"2024-08-15T17:59:21.016071+0000","last_peered":"2024-08-15T17:59:21.016071+0000","last_clean":"2024-08-15T17:59:21.016071+0000","last_became_active":"2024-08-15T17:59:21.007584+0000","last_became_peered":"2024-08-15T17:59:21.007584+0000","last_unstale":"2024-08-15T17:59:21.016071+0000","last_undegraded":"2024-08-15T17:59:21.016071+0000","last_fullsized":"2024-08-15T17:59:21.016071+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:21:08.329258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003903+0000","last_change":"2024-08-15T17:59:21.003903+0000","last_active":"2024-08-15T17:59:21.003903+0000","last_peered":"2024-08-15T17:59:21.003903+0000","last_clean":"2024-08-15T17:59:21.003903+0000","last_became_active":"2024-08-15T17:59:21.003517+0000","last_became_peered":"2024-08-15T17:59:21.003517+0000","last_unstale":"2024-08-15T17:59:21.003903+0000","last_undegraded":"2024-08-15T17:59:21.003903+0000","last_fullsized":"2024-08-15T17:59:21.003903+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:33:33.720588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016179+0000","last_change":"2024-08-15T17:59:21.011077+0000","last_active":"2024-08-15T17:59:21.016179+0000","last_peered":"2024-08-15T17:59:21.016179+0000","last_clean":"2024-08-15T17:59:21.016179+0000","last_became_active":"2024-08-15T17:59:21.008866+0000","last_became_peered":"2024-08-15T17:59:21.008866+0000","last_unstale":"2024-08-15T17:59:21.016179+0000","last_undegraded":"2024-08-15T17:59:21.016179+0000","last_fullsized":"2024-08-15T17:59:21.016179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:24:33.139116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004990+0000","last_change":"2024-08-15T17:59:21.004990+0000","last_active":"2024-08-15T17:59:21.004990+0000","last_peered":"2024-08-15T17:59:21.004990+0000","last_clean":"2024-08-15T17:59:21.004990+0000","last_became_active":"2024-08-15T17:59:21.004702+0000","last_became_peered":"2024-08-15T17:59:21.004702+0000","last_unstale":"2024-08-15T17:59:21.004990+0000","last_undegraded":"2024-08-15T17:59:21.004990+0000","last_fullsized":"2024-08-15T17:59:21.004990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:00:28.528787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003768+0000","last_change":"2024-08-15T17:59:21.003768+0000","last_active":"2024-08-15T17:59:21.003768+0000","last_peered":"2024-08-15T17:59:21.003768+0000","last_clean":"2024-08-15T17:59:21.003768+0000","last_became_active":"2024-08-15T17:59:21.003400+0000","last_became_peered":"2024-08-15T17:59:21.003400+0000","last_unstale":"2024-08-15T17:59:21.003768+0000","last_undegraded":"2024-08-15T17:59:21.003768+0000","last_fullsized":"2024-08-15T17:59:21.003768+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:52:47.789349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.008083+0000","last_change":"2024-08-15T17:59:21.008083+0000","last_active":"2024-08-15T17:59:21.008083+0000","last_peered":"2024-08-15T17:59:21.008083+0000","last_clean":"2024-08-15T17:59:21.008083+0000","last_became_active":"2024-08-15T17:59:21.007767+0000","last_became_peered":"2024-08-15T17:59:21.007767+0000","last_unstale":"2024-08-15T17:59:21.008083+0000","last_undegraded":"2024-08-15T17:59:21.008083+0000","last_fullsized":"2024-08-15T17:59:21.008083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:31:39.516975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016181+0000","last_change":"2024-08-15T17:59:21.010373+0000","last_active":"2024-08-15T17:59:21.016181+0000","last_peered":"2024-08-15T17:59:21.016181+0000","last_clean":"2024-08-15T17:59:21.016181+0000","last_became_active":"2024-08-15T17:59:21.010073+0000","last_became_peered":"2024-08-15T17:59:21.010073+0000","last_unstale":"2024-08-15T17:59:21.016181+0000","last_undegraded":"2024-08-15T17:59:21.016181+0000","last_fullsized":"2024-08-15T17:59:21.016181+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:48:44.709713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003043+0000","last_change":"2024-08-15T17:59:21.003043+0000","last_active":"2024-08-15T17:59:21.003043+0000","last_peered":"2024-08-15T17:59:21.003043+0000","last_clean":"2024-08-15T17:59:21.003043+0000","last_became_active":"2024-08-15T17:59:21.002267+0000","last_became_peered":"2024-08-15T17:59:21.002267+0000","last_unstale":"2024-08-15T17:59:21.003043+0000","last_undegraded":"2024-08-15T17:59:21.003043+0000","last_fullsized":"2024-08-15T17:59:21.003043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:18:33.342879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004423+0000","last_change":"2024-08-15T17:59:20.996955+0000","last_active":"2024-08-15T17:59:21.004423+0000","last_peered":"2024-08-15T17:59:21.004423+0000","last_clean":"2024-08-15T17:59:21.004423+0000","last_became_active":"2024-08-15T17:59:20.996652+0000","last_became_peered":"2024-08-15T17:59:20.996652+0000","last_unstale":"2024-08-15T17:59:21.004423+0000","last_undegraded":"2024-08-15T17:59:21.004423+0000","last_fullsized":"2024-08-15T17:59:21.004423+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:41.005396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.017799+0000","last_change":"2024-08-15T17:59:19.992199+0000","last_active":"2024-08-15T17:59:21.017799+0000","last_peered":"2024-08-15T17:59:21.017799+0000","last_clean":"2024-08-15T17:59:21.017799+0000","last_became_active":"2024-08-15T17:59:19.991914+0000","last_became_peered":"2024-08-15T17:59:19.991914+0000","last_unstale":"2024-08-15T17:59:21.017799+0000","last_undegraded":"2024-08-15T17:59:21.017799+0000","last_fullsized":"2024-08-15T17:59:21.017799+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:29:17.090193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.013746+0000","last_change":"2024-08-15T17:59:21.000563+0000","last_active":"2024-08-15T17:59:21.013746+0000","last_peered":"2024-08-15T17:59:21.013746+0000","last_clean":"2024-08-15T17:59:21.013746+0000","last_became_active":"2024-08-15T17:59:20.999966+0000","last_became_peered":"2024-08-15T17:59:20.999966+0000","last_unstale":"2024-08-15T17:59:21.013746+0000","last_undegraded":"2024-08-15T17:59:21.013746+0000","last_fullsized":"2024-08-15T17:59:21.013746+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:30:10.217672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.017920+0000","last_change":"2024-08-15T17:59:20.006977+0000","last_active":"2024-08-15T17:59:21.017920+0000","last_peered":"2024-08-15T17:59:21.017920+0000","last_clean":"2024-08-15T17:59:21.017920+0000","last_became_active":"2024-08-15T17:59:20.006674+0000","last_became_peered":"2024-08-15T17:59:20.006674+0000","last_unstale":"2024-08-15T17:59:21.017920+0000","last_undegraded":"2024-08-15T17:59:21.017920+0000","last_fullsized":"2024-08-15T17:59:21.017920+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:02:24.732030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003882+0000","last_change":"2024-08-15T17:59:21.003882+0000","last_active":"2024-08-15T17:59:21.003882+0000","last_peered":"2024-08-15T17:59:21.003882+0000","last_clean":"2024-08-15T17:59:21.003882+0000","last_became_active":"2024-08-15T17:59:21.000651+0000","last_became_peered":"2024-08-15T17:59:21.000651+0000","last_unstale":"2024-08-15T17:59:21.003882+0000","last_undegraded":"2024-08-15T17:59:21.003882+0000","last_fullsized":"2024-08-15T17:59:21.003882+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:26:32.993579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.018217+0000","last_change":"2024-08-15T17:59:20.016257+0000","last_active":"2024-08-15T17:59:21.018217+0000","last_peered":"2024-08-15T17:59:21.018217+0000","last_clean":"2024-08-15T17:59:21.018217+0000","last_became_active":"2024-08-15T17:59:20.015878+0000","last_became_peered":"2024-08-15T17:59:20.015878+0000","last_unstale":"2024-08-15T17:59:21.018217+0000","last_undegraded":"2024-08-15T17:59:21.018217+0000","last_fullsized":"2024-08-15T17:59:21.018217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:07:52.921234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016804+0000","last_change":"2024-08-15T17:59:21.014608+0000","last_active":"2024-08-15T17:59:21.016804+0000","last_peered":"2024-08-15T17:59:21.016804+0000","last_clean":"2024-08-15T17:59:21.016804+0000","last_became_active":"2024-08-15T17:59:21.010794+0000","last_became_peered":"2024-08-15T17:59:21.010794+0000","last_unstale":"2024-08-15T17:59:21.016804+0000","last_undegraded":"2024-08-15T17:59:21.016804+0000","last_fullsized":"2024-08-15T17:59:21.016804+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:09:21.391162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.017043+0000","last_change":"2024-08-15T17:59:20.001118+0000","last_active":"2024-08-15T17:59:21.017043+0000","last_peered":"2024-08-15T17:59:21.017043+0000","last_clean":"2024-08-15T17:59:21.017043+0000","last_became_active":"2024-08-15T17:59:20.000811+0000","last_became_peered":"2024-08-15T17:59:20.000811+0000","last_unstale":"2024-08-15T17:59:21.017043+0000","last_undegraded":"2024-08-15T17:59:21.017043+0000","last_fullsized":"2024-08-15T17:59:21.017043+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:11.776526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.000931+0000","last_change":"2024-08-15T17:59:21.000931+0000","last_active":"2024-08-15T17:59:21.000931+0000","last_peered":"2024-08-15T17:59:21.000931+0000","last_clean":"2024-08-15T17:59:21.000931+0000","last_became_active":"2024-08-15T17:59:21.000655+0000","last_became_peered":"2024-08-15T17:59:21.000655+0000","last_unstale":"2024-08-15T17:59:21.000931+0000","last_undegraded":"2024-08-15T17:59:21.000931+0000","last_fullsized":"2024-08-15T17:59:21.000931+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:05:35.938370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.987678+0000","last_change":"2024-08-15T17:59:19.999710+0000","last_active":"2024-08-15T17:59:20.987678+0000","last_peered":"2024-08-15T17:59:20.987678+0000","last_clean":"2024-08-15T17:59:20.987678+0000","last_became_active":"2024-08-15T17:59:19.999414+0000","last_became_peered":"2024-08-15T17:59:19.999414+0000","last_unstale":"2024-08-15T17:59:20.987678+0000","last_undegraded":"2024-08-15T17:59:20.987678+0000","last_fullsized":"2024-08-15T17:59:20.987678+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:34:14.250494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.005784+0000","last_change":"2024-08-15T17:59:21.005784+0000","last_active":"2024-08-15T17:59:21.005784+0000","last_peered":"2024-08-15T17:59:21.005784+0000","last_clean":"2024-08-15T17:59:21.005784+0000","last_became_active":"2024-08-15T17:59:21.005502+0000","last_became_peered":"2024-08-15T17:59:21.005502+0000","last_unstale":"2024-08-15T17:59:21.005784+0000","last_undegraded":"2024-08-15T17:59:21.005784+0000","last_fullsized":"2024-08-15T17:59:21.005784+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:52:24.811634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.986692+0000","last_change":"2024-08-15T17:59:20.007939+0000","last_active":"2024-08-15T17:59:20.986692+0000","last_peered":"2024-08-15T17:59:20.986692+0000","last_clean":"2024-08-15T17:59:20.986692+0000","last_became_active":"2024-08-15T17:59:20.007589+0000","last_became_peered":"2024-08-15T17:59:20.007589+0000","last_unstale":"2024-08-15T17:59:20.986692+0000","last_undegraded":"2024-08-15T17:59:20.986692+0000","last_fullsized":"2024-08-15T17:59:20.986692+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:06:25.333482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.997432+0000","last_change":"2024-08-15T17:59:20.997432+0000","last_active":"2024-08-15T17:59:20.997432+0000","last_peered":"2024-08-15T17:59:20.997432+0000","last_clean":"2024-08-15T17:59:20.997432+0000","last_became_active":"2024-08-15T17:59:20.997121+0000","last_became_peered":"2024-08-15T17:59:20.997121+0000","last_unstale":"2024-08-15T17:59:20.997432+0000","last_undegraded":"2024-08-15T17:59:20.997432+0000","last_fullsized":"2024-08-15T17:59:20.997432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:38:59.592614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.016202+0000","last_change":"2024-08-15T17:59:20.010821+0000","last_active":"2024-08-15T17:59:20.016202+0000","last_peered":"2024-08-15T17:59:20.016202+0000","last_clean":"2024-08-15T17:59:20.016202+0000","last_became_active":"2024-08-15T17:59:20.010520+0000","last_became_peered":"2024-08-15T17:59:20.010520+0000","last_unstale":"2024-08-15T17:59:20.016202+0000","last_undegraded":"2024-08-15T17:59:20.016202+0000","last_fullsized":"2024-08-15T17:59:20.016202+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:20:00.781537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016334+0000","last_change":"2024-08-15T17:59:21.012454+0000","last_active":"2024-08-15T17:59:21.016334+0000","last_peered":"2024-08-15T17:59:21.016334+0000","last_clean":"2024-08-15T17:59:21.016334+0000","last_became_active":"2024-08-15T17:59:21.011904+0000","last_became_peered":"2024-08-15T17:59:21.011904+0000","last_unstale":"2024-08-15T17:59:21.016334+0000","last_undegraded":"2024-08-15T17:59:21.016334+0000","last_fullsized":"2024-08-15T17:59:21.016334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:00:04.873888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:19.993710+0000","last_change":"2024-08-15T17:59:19.988445+0000","last_active":"2024-08-15T17:59:19.993710+0000","last_peered":"2024-08-15T17:59:19.993710+0000","last_clean":"2024-08-15T17:59:19.993710+0000","last_became_active":"2024-08-15T17:59:19.987956+0000","last_became_peered":"2024-08-15T17:59:19.987956+0000","last_unstale":"2024-08-15T17:59:19.993710+0000","last_undegraded":"2024-08-15T17:59:19.993710+0000","last_fullsized":"2024-08-15T17:59:19.993710+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:21:50.783043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003008+0000","last_change":"2024-08-15T17:59:21.003008+0000","last_active":"2024-08-15T17:59:21.003008+0000","last_peered":"2024-08-15T17:59:21.003008+0000","last_clean":"2024-08-15T17:59:21.003008+0000","last_became_active":"2024-08-15T17:59:21.002745+0000","last_became_peered":"2024-08-15T17:59:21.002745+0000","last_unstale":"2024-08-15T17:59:21.003008+0000","last_undegraded":"2024-08-15T17:59:21.003008+0000","last_fullsized":"2024-08-15T17:59:21.003008+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:47:35.841722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.015583+0000","last_change":"2024-08-15T17:59:20.008416+0000","last_active":"2024-08-15T17:59:20.015583+0000","last_peered":"2024-08-15T17:59:20.015583+0000","last_clean":"2024-08-15T17:59:20.015583+0000","last_became_active":"2024-08-15T17:59:20.008139+0000","last_became_peered":"2024-08-15T17:59:20.008139+0000","last_unstale":"2024-08-15T17:59:20.015583+0000","last_undegraded":"2024-08-15T17:59:20.015583+0000","last_fullsized":"2024-08-15T17:59:20.015583+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:31:56.124147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.005761+0000","last_change":"2024-08-15T17:59:21.005761+0000","last_active":"2024-08-15T17:59:21.005761+0000","last_peered":"2024-08-15T17:59:21.005761+0000","last_clean":"2024-08-15T17:59:21.005761+0000","last_became_active":"2024-08-15T17:59:21.005094+0000","last_became_peered":"2024-08-15T17:59:21.005094+0000","last_unstale":"2024-08-15T17:59:21.005761+0000","last_undegraded":"2024-08-15T17:59:21.005761+0000","last_fullsized":"2024-08-15T17:59:21.005761+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:02.921095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.986464+0000","last_change":"2024-08-15T17:59:20.009580+0000","last_active":"2024-08-15T17:59:20.986464+0000","last_peered":"2024-08-15T17:59:20.986464+0000","last_clean":"2024-08-15T17:59:20.986464+0000","last_became_active":"2024-08-15T17:59:20.009285+0000","last_became_peered":"2024-08-15T17:59:20.009285+0000","last_unstale":"2024-08-15T17:59:20.986464+0000","last_undegraded":"2024-08-15T17:59:20.986464+0000","last_fullsized":"2024-08-15T17:59:20.986464+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:34:42.148689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002880+0000","last_change":"2024-08-15T17:59:21.002880+0000","last_active":"2024-08-15T17:59:21.002880+0000","last_peered":"2024-08-15T17:59:21.002880+0000","last_clean":"2024-08-15T17:59:21.002880+0000","last_became_active":"2024-08-15T17:59:21.002506+0000","last_became_peered":"2024-08-15T17:59:21.002506+0000","last_unstale":"2024-08-15T17:59:21.002880+0000","last_undegraded":"2024-08-15T17:59:21.002880+0000","last_fullsized":"2024-08-15T17:59:21.002880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:14:27.472146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016852+0000","last_change":"2024-08-15T17:59:19.999985+0000","last_active":"2024-08-15T17:59:21.016852+0000","last_peered":"2024-08-15T17:59:21.016852+0000","last_clean":"2024-08-15T17:59:21.016852+0000","last_became_active":"2024-08-15T17:59:19.999818+0000","last_became_peered":"2024-08-15T17:59:19.999818+0000","last_unstale":"2024-08-15T17:59:21.016852+0000","last_undegraded":"2024-08-15T17:59:21.016852+0000","last_fullsized":"2024-08-15T17:59:21.016852+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:16:36.508757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004125+0000","last_change":"2024-08-15T17:59:21.004125+0000","last_active":"2024-08-15T17:59:21.004125+0000","last_peered":"2024-08-15T17:59:21.004125+0000","last_clean":"2024-08-15T17:59:21.004125+0000","last_became_active":"2024-08-15T17:59:21.003865+0000","last_became_peered":"2024-08-15T17:59:21.003865+0000","last_unstale":"2024-08-15T17:59:21.004125+0000","last_undegraded":"2024-08-15T17:59:21.004125+0000","last_fullsized":"2024-08-15T17:59:21.004125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:14:27.472146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.019686+0000","last_change":"2024-08-15T17:59:20.016970+0000","last_active":"2024-08-15T17:59:21.019686+0000","last_peered":"2024-08-15T17:59:21.019686+0000","last_clean":"2024-08-15T17:59:21.019686+0000","last_became_active":"2024-08-15T17:59:20.016709+0000","last_became_peered":"2024-08-15T17:59:20.016709+0000","last_unstale":"2024-08-15T17:59:21.019686+0000","last_undegraded":"2024-08-15T17:59:21.019686+0000","last_fullsized":"2024-08-15T17:59:21.019686+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:18:33.377985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016588+0000","last_change":"2024-08-15T17:59:21.014942+0000","last_active":"2024-08-15T17:59:21.016588+0000","last_peered":"2024-08-15T17:59:21.016588+0000","last_clean":"2024-08-15T17:59:21.016588+0000","last_became_active":"2024-08-15T17:59:21.012197+0000","last_became_peered":"2024-08-15T17:59:21.012197+0000","last_unstale":"2024-08-15T17:59:21.016588+0000","last_undegraded":"2024-08-15T17:59:21.016588+0000","last_fullsized":"2024-08-15T17:59:21.016588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:58:36.501838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016119+0000","last_change":"2024-08-15T17:59:19.990509+0000","last_active":"2024-08-15T17:59:21.016119+0000","last_peered":"2024-08-15T17:59:21.016119+0000","last_clean":"2024-08-15T17:59:21.016119+0000","last_became_active":"2024-08-15T17:59:19.990207+0000","last_became_peered":"2024-08-15T17:59:19.990207+0000","last_unstale":"2024-08-15T17:59:21.016119+0000","last_undegraded":"2024-08-15T17:59:21.016119+0000","last_fullsized":"2024-08-15T17:59:21.016119+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:23:47.220851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.009311+0000","last_change":"2024-08-15T17:59:21.009311+0000","last_active":"2024-08-15T17:59:21.009311+0000","last_peered":"2024-08-15T17:59:21.009311+0000","last_clean":"2024-08-15T17:59:21.009311+0000","last_became_active":"2024-08-15T17:59:21.008998+0000","last_became_peered":"2024-08-15T17:59:21.008998+0000","last_unstale":"2024-08-15T17:59:21.009311+0000","last_undegraded":"2024-08-15T17:59:21.009311+0000","last_fullsized":"2024-08-15T17:59:21.009311+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:33:04.015051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002069+0000","last_change":"2024-08-15T17:59:20.006965+0000","last_active":"2024-08-15T17:59:21.002069+0000","last_peered":"2024-08-15T17:59:21.002069+0000","last_clean":"2024-08-15T17:59:21.002069+0000","last_became_active":"2024-08-15T17:59:20.006607+0000","last_became_peered":"2024-08-15T17:59:20.006607+0000","last_unstale":"2024-08-15T17:59:21.002069+0000","last_undegraded":"2024-08-15T17:59:21.002069+0000","last_fullsized":"2024-08-15T17:59:21.002069+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:09:00.911809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003202+0000","last_change":"2024-08-15T17:59:21.003202+0000","last_active":"2024-08-15T17:59:21.003202+0000","last_peered":"2024-08-15T17:59:21.003202+0000","last_clean":"2024-08-15T17:59:21.003202+0000","last_became_active":"2024-08-15T17:59:21.002546+0000","last_became_peered":"2024-08-15T17:59:21.002546+0000","last_unstale":"2024-08-15T17:59:21.003202+0000","last_undegraded":"2024-08-15T17:59:21.003202+0000","last_fullsized":"2024-08-15T17:59:21.003202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:32:46.254744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.015535+0000","last_change":"2024-08-15T17:59:20.008440+0000","last_active":"2024-08-15T17:59:20.015535+0000","last_peered":"2024-08-15T17:59:20.015535+0000","last_clean":"2024-08-15T17:59:20.015535+0000","last_became_active":"2024-08-15T17:59:20.008129+0000","last_became_peered":"2024-08-15T17:59:20.008129+0000","last_unstale":"2024-08-15T17:59:20.015535+0000","last_undegraded":"2024-08-15T17:59:20.015535+0000","last_fullsized":"2024-08-15T17:59:20.015535+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:39.995754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.009568+0000","last_change":"2024-08-15T17:59:21.001572+0000","last_active":"2024-08-15T17:59:21.009568+0000","last_peered":"2024-08-15T17:59:21.009568+0000","last_clean":"2024-08-15T17:59:21.009568+0000","last_became_active":"2024-08-15T17:59:21.001291+0000","last_became_peered":"2024-08-15T17:59:21.001291+0000","last_unstale":"2024-08-15T17:59:21.009568+0000","last_undegraded":"2024-08-15T17:59:21.009568+0000","last_fullsized":"2024-08-15T17:59:21.009568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:38:59.592614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.020179+0000","last_change":"2024-08-15T17:59:20.005726+0000","last_active":"2024-08-15T17:59:21.020179+0000","last_peered":"2024-08-15T17:59:21.020179+0000","last_clean":"2024-08-15T17:59:21.020179+0000","last_became_active":"2024-08-15T17:59:20.005441+0000","last_became_peered":"2024-08-15T17:59:20.005441+0000","last_unstale":"2024-08-15T17:59:21.020179+0000","last_undegraded":"2024-08-15T17:59:21.020179+0000","last_fullsized":"2024-08-15T17:59:21.020179+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:35:23.963840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.000227+0000","last_change":"2024-08-15T17:59:21.000227+0000","last_active":"2024-08-15T17:59:21.000227+0000","last_peered":"2024-08-15T17:59:21.000227+0000","last_clean":"2024-08-15T17:59:21.000227+0000","last_became_active":"2024-08-15T17:59:20.999745+0000","last_became_peered":"2024-08-15T17:59:20.999745+0000","last_unstale":"2024-08-15T17:59:21.000227+0000","last_undegraded":"2024-08-15T17:59:21.000227+0000","last_fullsized":"2024-08-15T17:59:21.000227+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:31:01.411361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.018763+0000","last_change":"2024-08-15T17:59:20.016865+0000","last_active":"2024-08-15T17:59:21.018763+0000","last_peered":"2024-08-15T17:59:21.018763+0000","last_clean":"2024-08-15T17:59:21.018763+0000","last_became_active":"2024-08-15T17:59:20.016552+0000","last_became_peered":"2024-08-15T17:59:20.016552+0000","last_unstale":"2024-08-15T17:59:21.018763+0000","last_undegraded":"2024-08-15T17:59:21.018763+0000","last_fullsized":"2024-08-15T17:59:21.018763+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:06:37.220646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003065+0000","last_change":"2024-08-15T17:59:21.003065+0000","last_active":"2024-08-15T17:59:21.003065+0000","last_peered":"2024-08-15T17:59:21.003065+0000","last_clean":"2024-08-15T17:59:21.003065+0000","last_became_active":"2024-08-15T17:59:21.002635+0000","last_became_peered":"2024-08-15T17:59:21.002635+0000","last_unstale":"2024-08-15T17:59:21.003065+0000","last_undegraded":"2024-08-15T17:59:21.003065+0000","last_fullsized":"2024-08-15T17:59:21.003065+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:33:36.843048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016232+0000","last_change":"2024-08-15T17:59:19.990857+0000","last_active":"2024-08-15T17:59:21.016232+0000","last_peered":"2024-08-15T17:59:21.016232+0000","last_clean":"2024-08-15T17:59:21.016232+0000","last_became_active":"2024-08-15T17:59:19.990561+0000","last_became_peered":"2024-08-15T17:59:19.990561+0000","last_unstale":"2024-08-15T17:59:21.016232+0000","last_undegraded":"2024-08-15T17:59:21.016232+0000","last_fullsized":"2024-08-15T17:59:21.016232+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:52:21.522584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.016638+0000","last_change":"2024-08-15T17:59:21.008754+0000","last_active":"2024-08-15T17:59:21.016638+0000","last_peered":"2024-08-15T17:59:21.016638+0000","last_clean":"2024-08-15T17:59:21.016638+0000","last_became_active":"2024-08-15T17:59:21.008461+0000","last_became_peered":"2024-08-15T17:59:21.008461+0000","last_unstale":"2024-08-15T17:59:21.016638+0000","last_undegraded":"2024-08-15T17:59:21.016638+0000","last_fullsized":"2024-08-15T17:59:21.016638+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:26:53.696075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.986132+0000","last_change":"2024-08-15T17:59:19.992818+0000","last_active":"2024-08-15T17:59:20.986132+0000","last_peered":"2024-08-15T17:59:20.986132+0000","last_clean":"2024-08-15T17:59:20.986132+0000","last_became_active":"2024-08-15T17:59:19.992480+0000","last_became_peered":"2024-08-15T17:59:19.992480+0000","last_unstale":"2024-08-15T17:59:20.986132+0000","last_undegraded":"2024-08-15T17:59:20.986132+0000","last_fullsized":"2024-08-15T17:59:20.986132+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:20:01.760150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.997507+0000","last_change":"2024-08-15T17:59:20.997507+0000","last_active":"2024-08-15T17:59:20.997507+0000","last_peered":"2024-08-15T17:59:20.997507+0000","last_clean":"2024-08-15T17:59:20.997507+0000","last_became_active":"2024-08-15T17:59:20.997178+0000","last_became_peered":"2024-08-15T17:59:20.997178+0000","last_unstale":"2024-08-15T17:59:20.997507+0000","last_undegraded":"2024-08-15T17:59:20.997507+0000","last_fullsized":"2024-08-15T17:59:20.997507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:20:02.769792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.007273+0000","last_change":"2024-08-15T17:59:19.999638+0000","last_active":"2024-08-15T17:59:20.007273+0000","last_peered":"2024-08-15T17:59:20.007273+0000","last_clean":"2024-08-15T17:59:20.007273+0000","last_became_active":"2024-08-15T17:59:19.999395+0000","last_became_peered":"2024-08-15T17:59:19.999395+0000","last_unstale":"2024-08-15T17:59:20.007273+0000","last_undegraded":"2024-08-15T17:59:20.007273+0000","last_fullsized":"2024-08-15T17:59:20.007273+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:04:43.875545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.996319+0000","last_change":"2024-08-15T17:59:20.996319+0000","last_active":"2024-08-15T17:59:20.996319+0000","last_peered":"2024-08-15T17:59:20.996319+0000","last_clean":"2024-08-15T17:59:20.996319+0000","last_became_active":"2024-08-15T17:59:20.995697+0000","last_became_peered":"2024-08-15T17:59:20.995697+0000","last_unstale":"2024-08-15T17:59:20.996319+0000","last_undegraded":"2024-08-15T17:59:20.996319+0000","last_fullsized":"2024-08-15T17:59:20.996319+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:52:22.532226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.017965+0000","last_change":"2024-08-15T17:59:20.011231+0000","last_active":"2024-08-15T17:59:21.017965+0000","last_peered":"2024-08-15T17:59:21.017965+0000","last_clean":"2024-08-15T17:59:21.017965+0000","last_became_active":"2024-08-15T17:59:20.010939+0000","last_became_peered":"2024-08-15T17:59:20.010939+0000","last_unstale":"2024-08-15T17:59:21.017965+0000","last_undegraded":"2024-08-15T17:59:21.017965+0000","last_fullsized":"2024-08-15T17:59:21.017965+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:59:42.964222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002464+0000","last_change":"2024-08-15T17:59:21.002464+0000","last_active":"2024-08-15T17:59:21.002464+0000","last_peered":"2024-08-15T17:59:21.002464+0000","last_clean":"2024-08-15T17:59:21.002464+0000","last_became_active":"2024-08-15T17:59:21.002069+0000","last_became_peered":"2024-08-15T17:59:21.002069+0000","last_unstale":"2024-08-15T17:59:21.002464+0000","last_undegraded":"2024-08-15T17:59:21.002464+0000","last_fullsized":"2024-08-15T17:59:21.002464+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:30.986582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.994584+0000","last_change":"2024-08-15T17:59:19.999099+0000","last_active":"2024-08-15T17:59:20.994584+0000","last_peered":"2024-08-15T17:59:20.994584+0000","last_clean":"2024-08-15T17:59:20.994584+0000","last_became_active":"2024-08-15T17:59:19.998790+0000","last_became_peered":"2024-08-15T17:59:19.998790+0000","last_unstale":"2024-08-15T17:59:20.994584+0000","last_undegraded":"2024-08-15T17:59:20.994584+0000","last_fullsized":"2024-08-15T17:59:20.994584+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:52:04.693417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002740+0000","last_change":"2024-08-15T17:59:21.002740+0000","last_active":"2024-08-15T17:59:21.002740+0000","last_peered":"2024-08-15T17:59:21.002740+0000","last_clean":"2024-08-15T17:59:21.002740+0000","last_became_active":"2024-08-15T17:59:21.002240+0000","last_became_peered":"2024-08-15T17:59:21.002240+0000","last_unstale":"2024-08-15T17:59:21.002740+0000","last_undegraded":"2024-08-15T17:59:21.002740+0000","last_fullsized":"2024-08-15T17:59:21.002740+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:06:26.343124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.001827+0000","last_change":"2024-08-15T17:59:19.997704+0000","last_active":"2024-08-15T17:59:20.001827+0000","last_peered":"2024-08-15T17:59:20.001827+0000","last_clean":"2024-08-15T17:59:20.001827+0000","last_became_active":"2024-08-15T17:59:19.997413+0000","last_became_peered":"2024-08-15T17:59:19.997413+0000","last_unstale":"2024-08-15T17:59:20.001827+0000","last_undegraded":"2024-08-15T17:59:20.001827+0000","last_fullsized":"2024-08-15T17:59:20.001827+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:38:21.644793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.010619+0000","last_change":"2024-08-15T17:59:21.010619+0000","last_active":"2024-08-15T17:59:21.010619+0000","last_peered":"2024-08-15T17:59:21.010619+0000","last_clean":"2024-08-15T17:59:21.010619+0000","last_became_active":"2024-08-15T17:59:21.010311+0000","last_became_peered":"2024-08-15T17:59:21.010311+0000","last_unstale":"2024-08-15T17:59:21.010619+0000","last_undegraded":"2024-08-15T17:59:21.010619+0000","last_fullsized":"2024-08-15T17:59:21.010619+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:26:33.249200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.004713+0000","last_change":"2024-08-15T17:59:20.000815+0000","last_active":"2024-08-15T17:59:20.004713+0000","last_peered":"2024-08-15T17:59:20.004713+0000","last_clean":"2024-08-15T17:59:20.004713+0000","last_became_active":"2024-08-15T17:59:20.000540+0000","last_became_peered":"2024-08-15T17:59:20.000540+0000","last_unstale":"2024-08-15T17:59:20.004713+0000","last_undegraded":"2024-08-15T17:59:20.004713+0000","last_fullsized":"2024-08-15T17:59:20.004713+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:00:59.526752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.007597+0000","last_change":"2024-08-15T17:59:21.007597+0000","last_active":"2024-08-15T17:59:21.007597+0000","last_peered":"2024-08-15T17:59:21.007597+0000","last_clean":"2024-08-15T17:59:21.007597+0000","last_became_active":"2024-08-15T17:59:21.007314+0000","last_became_peered":"2024-08-15T17:59:21.007314+0000","last_unstale":"2024-08-15T17:59:21.007597+0000","last_undegraded":"2024-08-15T17:59:21.007597+0000","last_fullsized":"2024-08-15T17:59:21.007597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:38:22.654435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.020691+0000","last_change":"2024-08-15T17:59:20.005291+0000","last_active":"2024-08-15T17:59:21.020691+0000","last_peered":"2024-08-15T17:59:21.020691+0000","last_clean":"2024-08-15T17:59:21.020691+0000","last_became_active":"2024-08-15T17:59:20.004900+0000","last_became_peered":"2024-08-15T17:59:20.004900+0000","last_unstale":"2024-08-15T17:59:21.020691+0000","last_undegraded":"2024-08-15T17:59:21.020691+0000","last_fullsized":"2024-08-15T17:59:21.020691+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:43:52.441863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003858+0000","last_change":"2024-08-15T17:59:21.003858+0000","last_active":"2024-08-15T17:59:21.003858+0000","last_peered":"2024-08-15T17:59:21.003858+0000","last_clean":"2024-08-15T17:59:21.003858+0000","last_became_active":"2024-08-15T17:59:21.003576+0000","last_became_peered":"2024-08-15T17:59:21.003576+0000","last_unstale":"2024-08-15T17:59:21.003858+0000","last_undegraded":"2024-08-15T17:59:21.003858+0000","last_fullsized":"2024-08-15T17:59:21.003858+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:36:38.729269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.986536+0000","last_change":"2024-08-15T17:59:20.011484+0000","last_active":"2024-08-15T17:59:20.986536+0000","last_peered":"2024-08-15T17:59:20.986536+0000","last_clean":"2024-08-15T17:59:20.986536+0000","last_became_active":"2024-08-15T17:59:20.011164+0000","last_became_peered":"2024-08-15T17:59:20.011164+0000","last_unstale":"2024-08-15T17:59:20.986536+0000","last_undegraded":"2024-08-15T17:59:20.986536+0000","last_fullsized":"2024-08-15T17:59:20.986536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:10:46.666183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.009384+0000","last_change":"2024-08-15T17:59:21.003804+0000","last_active":"2024-08-15T17:59:21.009384+0000","last_peered":"2024-08-15T17:59:21.009384+0000","last_clean":"2024-08-15T17:59:21.009384+0000","last_became_active":"2024-08-15T17:59:21.003448+0000","last_became_peered":"2024-08-15T17:59:21.003448+0000","last_unstale":"2024-08-15T17:59:21.009384+0000","last_undegraded":"2024-08-15T17:59:21.009384+0000","last_fullsized":"2024-08-15T17:59:21.009384+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:07:53.930876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002553+0000","last_change":"2024-08-15T17:59:20.010613+0000","last_active":"2024-08-15T17:59:21.002553+0000","last_peered":"2024-08-15T17:59:21.002553+0000","last_clean":"2024-08-15T17:59:21.002553+0000","last_became_active":"2024-08-15T17:59:20.010288+0000","last_became_peered":"2024-08-15T17:59:20.010288+0000","last_unstale":"2024-08-15T17:59:21.002553+0000","last_undegraded":"2024-08-15T17:59:21.002553+0000","last_fullsized":"2024-08-15T17:59:21.002553+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:05:39.068362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.013695+0000","last_change":"2024-08-15T17:59:21.007020+0000","last_active":"2024-08-15T17:59:21.013695+0000","last_peered":"2024-08-15T17:59:21.013695+0000","last_clean":"2024-08-15T17:59:21.013695+0000","last_became_active":"2024-08-15T17:59:21.006713+0000","last_became_peered":"2024-08-15T17:59:21.006713+0000","last_unstale":"2024-08-15T17:59:21.013695+0000","last_undegraded":"2024-08-15T17:59:21.013695+0000","last_fullsized":"2024-08-15T17:59:21.013695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:31:01.411361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.985601+0000","last_change":"2024-08-15T17:59:20.004178+0000","last_active":"2024-08-15T17:59:20.985601+0000","last_peered":"2024-08-15T17:59:20.985601+0000","last_clean":"2024-08-15T17:59:20.985601+0000","last_became_active":"2024-08-15T17:59:20.003882+0000","last_became_peered":"2024-08-15T17:59:20.003882+0000","last_unstale":"2024-08-15T17:59:20.985601+0000","last_undegraded":"2024-08-15T17:59:20.985601+0000","last_fullsized":"2024-08-15T17:59:20.985601+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:14:30.395953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.015866+0000","last_change":"2024-08-15T17:59:21.009520+0000","last_active":"2024-08-15T17:59:21.015866+0000","last_peered":"2024-08-15T17:59:21.015866+0000","last_clean":"2024-08-15T17:59:21.015866+0000","last_became_active":"2024-08-15T17:59:21.009012+0000","last_became_peered":"2024-08-15T17:59:21.009012+0000","last_unstale":"2024-08-15T17:59:21.015866+0000","last_undegraded":"2024-08-15T17:59:21.015866+0000","last_fullsized":"2024-08-15T17:59:21.015866+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:31:57.133789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.002204+0000","last_change":"2024-08-15T17:59:20.005853+0000","last_active":"2024-08-15T17:59:21.002204+0000","last_peered":"2024-08-15T17:59:21.002204+0000","last_clean":"2024-08-15T17:59:21.002204+0000","last_became_active":"2024-08-15T17:59:20.005555+0000","last_became_peered":"2024-08-15T17:59:20.005555+0000","last_unstale":"2024-08-15T17:59:21.002204+0000","last_undegraded":"2024-08-15T17:59:21.002204+0000","last_fullsized":"2024-08-15T17:59:21.002204+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:52:04.693417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.003843+0000","last_change":"2024-08-15T17:59:21.003843+0000","last_active":"2024-08-15T17:59:21.003843+0000","last_peered":"2024-08-15T17:59:21.003843+0000","last_clean":"2024-08-15T17:59:21.003843+0000","last_became_active":"2024-08-15T17:59:21.003533+0000","last_became_peered":"2024-08-15T17:59:21.003533+0000","last_unstale":"2024-08-15T17:59:21.003843+0000","last_undegraded":"2024-08-15T17:59:21.003843+0000","last_fullsized":"2024-08-15T17:59:21.003843+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:47:35.841722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:19.998991+0000","last_change":"2024-08-15T17:59:19.992086+0000","last_active":"2024-08-15T17:59:19.998991+0000","last_peered":"2024-08-15T17:59:19.998991+0000","last_clean":"2024-08-15T17:59:19.998991+0000","last_became_active":"2024-08-15T17:59:19.991727+0000","last_became_peered":"2024-08-15T17:59:19.991727+0000","last_unstale":"2024-08-15T17:59:19.998991+0000","last_undegraded":"2024-08-15T17:59:19.998991+0000","last_fullsized":"2024-08-15T17:59:19.998991+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:18:57.157654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.004762+0000","last_change":"2024-08-15T17:59:21.004762+0000","last_active":"2024-08-15T17:59:21.004762+0000","last_peered":"2024-08-15T17:59:21.004762+0000","last_clean":"2024-08-15T17:59:21.004762+0000","last_became_active":"2024-08-15T17:59:21.003777+0000","last_became_peered":"2024-08-15T17:59:21.003777+0000","last_unstale":"2024-08-15T17:59:21.004762+0000","last_undegraded":"2024-08-15T17:59:21.004762+0000","last_fullsized":"2024-08-15T17:59:21.004762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:30:10.217672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.015698+0000","last_change":"2024-08-15T17:59:20.008771+0000","last_active":"2024-08-15T17:59:20.015698+0000","last_peered":"2024-08-15T17:59:20.015698+0000","last_clean":"2024-08-15T17:59:20.015698+0000","last_became_active":"2024-08-15T17:59:20.008494+0000","last_became_peered":"2024-08-15T17:59:20.008494+0000","last_unstale":"2024-08-15T17:59:20.015698+0000","last_undegraded":"2024-08-15T17:59:20.015698+0000","last_fullsized":"2024-08-15T17:59:20.015698+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:15:04.414906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:21.008525+0000","last_change":"2024-08-15T17:59:21.008525+0000","last_active":"2024-08-15T17:59:21.008525+0000","last_peered":"2024-08-15T17:59:21.008525+0000","last_clean":"2024-08-15T17:59:21.008525+0000","last_became_active":"2024-08-15T17:59:21.008253+0000","last_became_peered":"2024-08-15T17:59:21.008253+0000","last_unstale":"2024-08-15T17:59:21.008525+0000","last_undegraded":"2024-08-15T17:59:21.008525+0000","last_fullsized":"2024-08-15T17:59:21.008525+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:19.965245+0000","last_clean_scrub_stamp":"2024-08-15T17:59:19.965245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:58:21.676184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:59:20.994165+0000","last_change":"2024-08-15T17:59:19.993347+0000","last_active":"2024-08-15T17:59:20.994165+0000","last_peered":"2024-08-15T17:59:20.994165+0000","last_clean":"2024-08-15T17:59:20.994165+0000","last_became_active":"2024-08-15T17:59:19.993017+0000","last_became_peered":"2024-08-15T17:59:19.993017+0000","last_unstale":"2024-08-15T17:59:20.994165+0000","last_undegraded":"2024-08-15T17:59:20.994165+0000","last_fullsized":"2024-08-15T17:59:20.994165+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:59:18.955603+0000","last_clean_scrub_stamp":"2024-08-15T17:59:18.955603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:29.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:59:19.998192+0000","last_change":"2024-08-15T17:58:24.394014+0000","last_active":"2024-08-15T17:59:19.998192+0000","last_peered":"2024-08-15T17:59:19.998192+0000","last_clean":"2024-08-15T17:59:19.998192+0000","last_became_active":"2024-08-15T17:58:24.391735+0000","last_became_peered":"2024-08-15T17:58:24.391735+0000","last_unstale":"2024-08-15T17:59:19.998192+0000","last_undegraded":"2024-08-15T17:59:19.998192+0000","last_fullsized":"2024-08-15T17:59:19.998192+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_clean_scrub_stamp":"2024-08-15T17:58:22.996335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:31.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302440,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069400,"statfs":{"total":96636764160,"available":96327065600,"internally_reserved":0,"allocated":937984,"data_stored":720031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297708,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074132,"statfs":{"total":96636764160,"available":96331911168,"internally_reserved":0,"allocated":360448,"data_stored":136465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]}]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":348160,"data_stored":129677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301856,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069984,"statfs":{"total":96636764160,"available":96327663616,"internally_reserved":0,"allocated":348160,"data_stored":130233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":356352,"data_stored":131147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301852,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069988,"statfs":{"total":96636764160,"available":96327667712,"internally_reserved":0,"allocated":344064,"data_stored":129643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":356352,"data_stored":131147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298268,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073572,"statfs":{"total":96636764160,"available":96331337728,"internally_reserved":0,"allocated":933888,"data_stored":719941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:59:25.348 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-08-15T17:59:25.348 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-08-15T17:59:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.473+0000 7fd22dbb6700 1 Processor -- start 2024-08-15T17:59:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.473+0000 7fd22dbb6700 1 -- start start 2024-08-15T17:59:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.473+0000 7fd22dbb6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd2280cb5b0 0x7fd2280cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.473+0000 7fd22dbb6700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2280c5ce0 con 0x7fd228159830 2024-08-15T17:59:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.473+0000 7fd22dbb6700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2280cbff0 con 0x7fd2280d0f60 2024-08-15T17:59:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.473+0000 7fd22dbb6700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd228159c50 con 0x7fd2280cb5b0 2024-08-15T17:59:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd2277fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd2280cb5b0 0x7fd2280cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd227fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd228159830 0x7fd2280ce070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59218/0 (socket says 172.21.15.38:59218) 2024-08-15T17:59:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd227fff700 1 -- 172.21.15.38:0/3866858694 learned_addr learned my addr 172.21.15.38:0/3866858694 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:25.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd2267fc700 1 -- 172.21.15.38:0/3866858694 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330942399 0 0) 0x7fd2280c5ce0 con 0x7fd228159830 2024-08-15T17:59:25.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd2267fc700 1 -- 172.21.15.38:0/3866858694 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd214003570 con 0x7fd228159830 2024-08-15T17:59:25.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd2267fc700 1 -- 172.21.15.38:0/3866858694 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947482179 0 0) 0x7fd2280cbff0 con 0x7fd2280d0f60 2024-08-15T17:59:25.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd2277fe700 1 -- 172.21.15.38:0/3866858694 >> v1:172.21.15.104:6789/0 conn(0x7fd2280d0f60 legacy=0x7fd228159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd2277fe700 1 -- 172.21.15.38:0/3866858694 >> v1:172.21.15.38:6789/0 conn(0x7fd228159830 legacy=0x7fd2280ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd2277fe700 1 -- 172.21.15.38:0/3866858694 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2280cc290 con 0x7fd2280cb5b0 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd2277fe700 1 --2- 172.21.15.38:0/3866858694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2280cb5b0 0x7fd2280cb980 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd2100098e0 tx=0x7fd2100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287e7ec2ef6f1a34 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd2267fc700 1 -- 172.21.15.38:0/3866858694 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000b640 con 0x7fd2280cb5b0 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd2267fc700 1 -- 172.21.15.38:0/3866858694 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21000b7a0 con 0x7fd2280cb5b0 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd2267fc700 1 -- 172.21.15.38:0/3866858694 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000bb40 con 0x7fd2280cb5b0 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd22dbb6700 1 -- 172.21.15.38:0/3866858694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2280cb5b0 msgr2=0x7fd2280cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd22dbb6700 1 --2- 172.21.15.38:0/3866858694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2280cb5b0 0x7fd2280cb980 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd2100098e0 tx=0x7fd2100091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd22dbb6700 1 -- 172.21.15.38:0/3866858694 shutdown_connections 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd22dbb6700 1 --2- 172.21.15.38:0/3866858694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2280cb5b0 0x7fd2280cb980 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd22dbb6700 1 -- 172.21.15.38:0/3866858694 >> 172.21.15.38:0/3866858694 conn(0x7fd228067590 msgr2=0x7fd2280d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd22dbb6700 1 -- 172.21.15.38:0/3866858694 shutdown_connections 2024-08-15T17:59:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.477+0000 7fd22dbb6700 1 -- 172.21.15.38:0/3866858694 wait complete. 2024-08-15T17:59:25.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd22dbb6700 1 Processor -- start 2024-08-15T17:59:25.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd22dbb6700 1 -- start start 2024-08-15T17:59:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd22dbb6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2280cb5b0 0x7fd228153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd22dbb6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2280d0f60 0x7fd2280b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd22dbb6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd228159830 0x7fd2280b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd22dbb6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280ca4a0 con 0x7fd2280d0f60 2024-08-15T17:59:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd22dbb6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280b4e20 con 0x7fd228159830 2024-08-15T17:59:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd22dbb6700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280b4210 con 0x7fd2280cb5b0 2024-08-15T17:59:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd226ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2280d0f60 0x7fd2280b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd226ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2280d0f60 0x7fd2280b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41802/0 (socket says 172.21.15.38:41802) 2024-08-15T17:59:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd226ffd700 1 -- 172.21.15.38:0/585941445 learned_addr learned my addr 172.21.15.38:0/585941445 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd227fff700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd228159830 0x7fd2280b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd2277fe700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2280cb5b0 0x7fd228153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd226ffd700 1 -- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2280cb5b0 msgr2=0x7fd228153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd226ffd700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2280cb5b0 0x7fd228153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd226ffd700 1 -- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd228159830 msgr2=0x7fd2280b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd226ffd700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd228159830 0x7fd2280b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd226ffd700 1 -- 172.21.15.38:0/585941445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210009420 con 0x7fd2280d0f60 2024-08-15T17:59:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd227fff700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd228159830 0x7fd2280b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:59:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd226ffd700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2280d0f60 0x7fd2280b52b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd21800c810 tx=0x7fd218007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:25.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd21f7fe700 1 -- 172.21.15.38:0/585941445 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2180105c0 con 0x7fd2280d0f60 2024-08-15T17:59:25.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd21f7fe700 1 -- 172.21.15.38:0/585941445 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd218010740 con 0x7fd2280d0f60 2024-08-15T17:59:25.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd21f7fe700 1 -- 172.21.15.38:0/585941445 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218010b00 con 0x7fd2280d0f60 2024-08-15T17:59:25.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.481+0000 7fd22dbb6700 1 -- 172.21.15.38:0/585941445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2280b44f0 con 0x7fd2280d0f60 2024-08-15T17:59:25.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.485+0000 7fd22dbb6700 1 -- 172.21.15.38:0/585941445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2280b4ab0 con 0x7fd2280d0f60 2024-08-15T17:59:25.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.485+0000 7fd22dbb6700 1 -- 172.21.15.38:0/585941445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2280645c0 con 0x7fd2280d0f60 2024-08-15T17:59:25.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.485+0000 7fd21f7fe700 1 -- 172.21.15.38:0/585941445 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd21801b3b0 con 0x7fd2280d0f60 2024-08-15T17:59:25.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.485+0000 7fd21f7fe700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd208060860 0x7fd208062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.485+0000 7fd21f7fe700 1 -- 172.21.15.38:0/585941445 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd21807c1f0 con 0x7fd2280d0f60 2024-08-15T17:59:25.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.489+0000 7fd2277fe700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd208060860 0x7fd208062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.489+0000 7fd2277fe700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd208060860 0x7fd208062d10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd2100093f0 tx=0x7fd210017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:25.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.493+0000 7fd21f7fe700 1 -- 172.21.15.38:0/585941445 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd21804c640 con 0x7fd2280d0f60 2024-08-15T17:59:25.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.721+0000 7fd22dbb6700 1 -- 172.21.15.38:0/585941445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd2280b4f60 con 0x7fd2280d0f60 2024-08-15T17:59:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.729+0000 7fd21f7fe700 1 -- 172.21.15.38:0/585941445 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd2180170e0 con 0x7fd2280d0f60 2024-08-15T17:59:25.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 -- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd208060860 msgr2=0x7fd208062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd208060860 0x7fd208062d10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd2100093f0 tx=0x7fd210017000 comp rx=0 tx=0).stop 2024-08-15T17:59:25.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 -- 172.21.15.38:0/585941445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2280d0f60 msgr2=0x7fd2280b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2280d0f60 0x7fd2280b52b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd21800c810 tx=0x7fd218007df0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 -- 172.21.15.38:0/585941445 shutdown_connections 2024-08-15T17:59:25.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd208060860 0x7fd208062d10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd228159830 0x7fd2280b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2280d0f60 0x7fd2280b52b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 --2- 172.21.15.38:0/585941445 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2280cb5b0 0x7fd228153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 -- 172.21.15.38:0/585941445 >> 172.21.15.38:0/585941445 conn(0x7fd228067590 msgr2=0x7fd2280692e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:25.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 -- 172.21.15.38:0/585941445 shutdown_connections 2024-08-15T17:59:25.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.733+0000 7fd22dbb6700 1 -- 172.21.15.38:0/585941445 wait complete. 2024-08-15T17:59:25.733 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-08-15T17:59:25.745 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:59:25.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f989caa1700 1 Processor -- start 2024-08-15T17:59:25.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f989caa1700 1 -- start start 2024-08-15T17:59:25.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f989caa1700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f98980d0010 0x7f98980d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f989caa1700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98980cf190 con 0x7f9898159040 2024-08-15T17:59:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f989caa1700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98980d09a0 con 0x7f989815cea0 2024-08-15T17:59:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f989caa1700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989815d9f0 con 0x7f98980d0010 2024-08-15T17:59:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f9896ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f98980d0010 0x7f98980d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f98977fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9898159040 0x7f98980cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59230/0 (socket says 172.21.15.38:59230) 2024-08-15T17:59:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f98977fe700 1 -- 172.21.15.38:0/5981804 learned_addr learned my addr 172.21.15.38:0/5981804 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f9896ffd700 1 -- 172.21.15.38:0/5981804 >> v1:172.21.15.104:6789/0 conn(0x7f989815cea0 legacy=0x7f989815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f9896ffd700 1 -- 172.21.15.38:0/5981804 >> v1:172.21.15.38:6789/0 conn(0x7f9898159040 legacy=0x7f98980cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f98967fc700 1 -- 172.21.15.38:0/5981804 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826995457 0 0) 0x7f98980cf190 con 0x7f9898159040 2024-08-15T17:59:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f9896ffd700 1 -- 172.21.15.38:0/5981804 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98980ce5e0 con 0x7f98980d0010 2024-08-15T17:59:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f9896ffd700 1 --2- 172.21.15.38:0/5981804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98980d0010 0x7f98980d0460 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f98800098e0 tx=0x7f98800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a05ea9dccfd51e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f98967fc700 1 -- 172.21.15.38:0/5981804 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880009e80 con 0x7f98980d0010 2024-08-15T17:59:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f98967fc700 1 -- 172.21.15.38:0/5981804 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988000b860 con 0x7f98980d0010 2024-08-15T17:59:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f98967fc700 1 -- 172.21.15.38:0/5981804 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000bbe0 con 0x7f98980d0010 2024-08-15T17:59:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f989caa1700 1 -- 172.21.15.38:0/5981804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98980d0010 msgr2=0x7f98980d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f989caa1700 1 --2- 172.21.15.38:0/5981804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98980d0010 0x7f98980d0460 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f98800098e0 tx=0x7f98800091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f989caa1700 1 -- 172.21.15.38:0/5981804 shutdown_connections 2024-08-15T17:59:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f989caa1700 1 --2- 172.21.15.38:0/5981804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98980d0010 0x7f98980d0460 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.853+0000 7f989caa1700 1 -- 172.21.15.38:0/5981804 >> 172.21.15.38:0/5981804 conn(0x7f9898068250 msgr2=0x7f9898068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 -- 172.21.15.38:0/5981804 shutdown_connections 2024-08-15T17:59:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 -- 172.21.15.38:0/5981804 wait complete. 2024-08-15T17:59:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 Processor -- start 2024-08-15T17:59:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 -- start start 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98980d0010 0x7f9898159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9898159040 0x7f98980b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f989815cea0 0x7f989816a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9898149650 con 0x7f9898159040 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989815b350 con 0x7f98980d0010 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989815aec0 con 0x7f989815cea0 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f9896ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9898159040 0x7f98980b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f9896ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9898159040 0x7f98980b1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41828/0 (socket says 172.21.15.38:41828) 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f9896ffd700 1 -- 172.21.15.38:0/2921121136 learned_addr learned my addr 172.21.15.38:0/2921121136 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f9897fff700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f989815cea0 0x7f989816a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f98977fe700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98980d0010 0x7f9898159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f9896ffd700 1 -- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f989815cea0 msgr2=0x7f989816a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f9896ffd700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f989815cea0 0x7f989816a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f9896ffd700 1 -- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98980d0010 msgr2=0x7f9898159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f9896ffd700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98980d0010 0x7f9898159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f9896ffd700 1 -- 172.21.15.38:0/2921121136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9880009420 con 0x7f9898159040 2024-08-15T17:59:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f9896ffd700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9898159040 0x7f98980b1cb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9898088ef0 tx=0x7f9880017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f988f7fe700 1 -- 172.21.15.38:0/2921121136 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000bbe0 con 0x7f9898159040 2024-08-15T17:59:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f988f7fe700 1 -- 172.21.15.38:0/2921121136 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880011420 con 0x7f9898159040 2024-08-15T17:59:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 -- 172.21.15.38:0/2921121136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989815a2b0 con 0x7f9898159040 2024-08-15T17:59:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f988f7fe700 1 -- 172.21.15.38:0/2921121136 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880011820 con 0x7f9898159040 2024-08-15T17:59:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.857+0000 7f989caa1700 1 -- 172.21.15.38:0/2921121136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989815a870 con 0x7f9898159040 2024-08-15T17:59:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.861+0000 7f989caa1700 1 -- 172.21.15.38:0/2921121136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989815b070 con 0x7f9898159040 2024-08-15T17:59:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.861+0000 7f988f7fe700 1 -- 172.21.15.38:0/2921121136 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9880011bd0 con 0x7f9898159040 2024-08-15T17:59:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.861+0000 7f988f7fe700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9878060890 0x7f9878062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.861+0000 7f988f7fe700 1 -- 172.21.15.38:0/2921121136 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f988007e4f0 con 0x7f9898159040 2024-08-15T17:59:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.861+0000 7f98977fe700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9878060890 0x7f9878062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.865+0000 7f98977fe700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9878060890 0x7f9878062d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9888000c00 tx=0x7f9888009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:25.869+0000 7f988f7fe700 1 -- 172.21.15.38:0/2921121136 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f989815b070 con 0x7f9898159040 2024-08-15T17:59:26.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.069+0000 7f989caa1700 1 -- 172.21.15.38:0/2921121136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f989815b030 con 0x7f9898159040 2024-08-15T17:59:26.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f988f7fe700 1 -- 172.21.15.38:0/2921121136 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f988004e8a0 con 0x7f9898159040 2024-08-15T17:59:26.071 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:26.071 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/373633111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":373633111},{"type":"v1","addr":"172.21.15.38:6833","nonce":373633111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/389886189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":389886189},{"type":"v1","addr":"172.21.15.38:6835","nonce":389886189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4981,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/875828080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":875828080},{"type":"v1","addr":"172.21.15.104:6835","nonce":875828080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:20.353438+0000","modified":"2024-08-15T17:59:24.015140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4999},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4999":{"gid":4999,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/1865762791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1865762791},{"type":"v1","addr":"172.21.15.104:6837","nonce":1865762791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-08-15T17:59:26.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 -- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9878060890 msgr2=0x7f9878062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9878060890 0x7f9878062d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9888000c00 tx=0x7f9888009040 comp rx=0 tx=0).stop 2024-08-15T17:59:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 -- 172.21.15.38:0/2921121136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9898159040 msgr2=0x7f98980b1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9898159040 0x7f98980b1cb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9898088ef0 tx=0x7f9880017000 comp rx=0 tx=0).stop 2024-08-15T17:59:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 -- 172.21.15.38:0/2921121136 shutdown_connections 2024-08-15T17:59:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9878060890 0x7f9878062d40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f989815cea0 0x7f989816a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9898159040 0x7f98980b1cb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 --2- 172.21.15.38:0/2921121136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98980d0010 0x7f9898159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 -- 172.21.15.38:0/2921121136 >> 172.21.15.38:0/2921121136 conn(0x7f9898068250 msgr2=0x7f9898152dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 -- 172.21.15.38:0/2921121136 shutdown_connections 2024-08-15T17:59:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.073+0000 7f989caa1700 1 -- 172.21.15.38:0/2921121136 wait complete. 2024-08-15T17:59:26.075 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 15 2024-08-15T17:59:26.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-15T17:59:20.353438+0000', 'modified': '2024-08-15T17:59:24.015140+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4999}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4999': {'gid': 4999, 'name': 'd', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6837/1865762791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 1865762791}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 1865762791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-08-15T17:59:26.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-15T17:59:26.488 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-08-15T17:59:26.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.609+0000 7f120f3b2700 1 Processor -- start 2024-08-15T17:59:26.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.609+0000 7f120f3b2700 1 -- start start 2024-08-15T17:59:26.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.609+0000 7f120f3b2700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f120806e9d0 0x7f120806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.609+0000 7f120f3b2700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f12080ab540 0x7f12080b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.609+0000 7f120f3b2700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12080aba70 con 0x7f120806df10 2024-08-15T17:59:26.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.609+0000 7f120f3b2700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12080abd50 con 0x7f12080ab540 2024-08-15T17:59:26.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.609+0000 7f120f3b2700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12080ac030 con 0x7f120806e9d0 2024-08-15T17:59:26.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120dbaf700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f12080ab540 0x7f12080b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120dbaf700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f12080ab540 0x7f12080b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35468/0 (socket says 172.21.15.38:35468) 2024-08-15T17:59:26.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120dbaf700 1 -- 172.21.15.38:0/4268422170 learned_addr learned my addr 172.21.15.38:0/4268422170 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:26.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120e3b0700 1 --2- 172.21.15.38:0/4268422170 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f120806e9d0 0x7f120806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120dbaf700 1 -- 172.21.15.38:0/4268422170 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f120806e9d0 msgr2=0x7f120806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:26.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120dbaf700 1 --2- 172.21.15.38:0/4268422170 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f120806e9d0 0x7f120806eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120dbaf700 1 -- 172.21.15.38:0/4268422170 >> v1:172.21.15.104:6789/0 conn(0x7f120806df10 legacy=0x7f1208155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120dbaf700 1 -- 172.21.15.38:0/4268422170 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080ac310 con 0x7f12080ab540 2024-08-15T17:59:26.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120e3b0700 1 --2- 172.21.15.38:0/4268422170 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f120806e9d0 0x7f120806eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:59:26.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120dbaf700 1 --2- 172.21.15.38:0/4268422170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12080ab540 0x7f12080b1320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0099b0 tx=0x7f11fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77763c71091d0e72 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:26.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120d3ae700 1 -- 172.21.15.38:0/4268422170 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00b860 con 0x7f12080ab540 2024-08-15T17:59:26.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120d3ae700 1 -- 172.21.15.38:0/4268422170 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc00b9c0 con 0x7f12080ab540 2024-08-15T17:59:26.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120d3ae700 1 -- 172.21.15.38:0/4268422170 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00bd60 con 0x7f12080ab540 2024-08-15T17:59:26.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120f3b2700 1 -- 172.21.15.38:0/4268422170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12080ab540 msgr2=0x7f12080b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120f3b2700 1 --2- 172.21.15.38:0/4268422170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12080ab540 0x7f12080b1320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0099b0 tx=0x7f11fc009250 comp rx=0 tx=0).stop 2024-08-15T17:59:26.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120f3b2700 1 -- 172.21.15.38:0/4268422170 shutdown_connections 2024-08-15T17:59:26.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120f3b2700 1 --2- 172.21.15.38:0/4268422170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12080ab540 0x7f12080b1320 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120f3b2700 1 --2- 172.21.15.38:0/4268422170 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f120806e9d0 0x7f120806eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120f3b2700 1 -- 172.21.15.38:0/4268422170 >> 172.21.15.38:0/4268422170 conn(0x7f12080678d0 msgr2=0x7f1208067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:26.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120f3b2700 1 -- 172.21.15.38:0/4268422170 shutdown_connections 2024-08-15T17:59:26.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.613+0000 7f120f3b2700 1 -- 172.21.15.38:0/4268422170 wait complete. 2024-08-15T17:59:26.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120f3b2700 1 Processor -- start 2024-08-15T17:59:26.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120f3b2700 1 -- start start 2024-08-15T17:59:26.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120f3b2700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f120806df10 0x7f1208135620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120f3b2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120806e9d0 0x7f1208147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120f3b2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12080ab540 0x7f120814b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120f3b2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1208135b60 con 0x7f120806e9d0 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120f3b2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1208137110 con 0x7f12080ab540 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120f3b2700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12081373f0 con 0x7f120806df10 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120dbaf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120806e9d0 0x7f1208147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120dbaf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120806e9d0 0x7f1208147b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35478/0 (socket says 172.21.15.38:35478) 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120dbaf700 1 -- 172.21.15.38:0/4059572751 learned_addr learned my addr 172.21.15.38:0/4059572751 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120e3b0700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f120806df10 0x7f1208135620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.617+0000 7f120ebb1700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12080ab540 0x7f120814b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f120dbaf700 1 -- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f120806df10 msgr2=0x7f1208135620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f120dbaf700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f120806df10 0x7f1208135620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f120dbaf700 1 -- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12080ab540 msgr2=0x7f120814b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f120dbaf700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12080ab540 0x7f120814b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f120dbaf700 1 -- 172.21.15.38:0/4059572751 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc0094f0 con 0x7f120806e9d0 2024-08-15T17:59:26.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f120dbaf700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120806e9d0 0x7f1208147b60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009290 tx=0x7f11fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f11fa7fc700 1 -- 172.21.15.38:0/4059572751 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc01a450 con 0x7f120806e9d0 2024-08-15T17:59:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f11fa7fc700 1 -- 172.21.15.38:0/4059572751 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc01a5b0 con 0x7f120806e9d0 2024-08-15T17:59:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f120f3b2700 1 -- 172.21.15.38:0/4059572751 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12081376d0 con 0x7f120806e9d0 2024-08-15T17:59:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f11fa7fc700 1 -- 172.21.15.38:0/4059572751 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc01a970 con 0x7f120806e9d0 2024-08-15T17:59:26.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f120f3b2700 1 -- 172.21.15.38:0/4059572751 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12081480a0 con 0x7f120806e9d0 2024-08-15T17:59:26.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.621+0000 7f11fa7fc700 1 -- 172.21.15.38:0/4059572751 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f11fc01ad20 con 0x7f120806e9d0 2024-08-15T17:59:26.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f11fa7fc700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f11ec060860 0x7f11ec062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f11fa7fc700 1 -- 172.21.15.38:0/4059572751 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f11fc07ea80 con 0x7f120806e9d0 2024-08-15T17:59:26.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f120e3b0700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f11ec060860 0x7f11ec062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f120e3b0700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f11ec060860 0x7f11ec062d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f120814f870 tx=0x7f1204006f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:26.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f12128c3740 1 Processor -- start 2024-08-15T17:59:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f12128c3740 1 -- start start 2024-08-15T17:59:26.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f12128c3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150cae0 0x17401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f12128c3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x1740710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f12128c3740 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x1557790 0x1743c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f12128c3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x16b8ba0 con 0x150cae0 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f12128c3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x16b4b40 con 0x150c710 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f12128c3740 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x167a0a0 con 0x1557790 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.625+0000 7f120e3b0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150cae0 0x17401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f120e3b0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150cae0 0x17401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35490/0 (socket says 172.21.15.38:35490) 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f120e3b0700 1 -- 172.21.15.38:0/170990106 learned_addr learned my addr 172.21.15.38:0/170990106 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f120dbaf700 1 --2- 172.21.15.38:0/170990106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x1740710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f120ebb1700 1 --2- 172.21.15.38:0/170990106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x1557790 0x1743c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f120e3b0700 1 -- 172.21.15.38:0/170990106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x1557790 msgr2=0x1743c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f120e3b0700 1 --2- 172.21.15.38:0/170990106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x1557790 0x1743c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f120e3b0700 1 -- 172.21.15.38:0/170990106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 msgr2=0x1740710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f120e3b0700 1 --2- 172.21.15.38:0/170990106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x1740710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f120e3b0700 1 -- 172.21.15.38:0/170990106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x15589e0 con 0x150cae0 2024-08-15T17:59:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f120e3b0700 1 --2- 172.21.15.38:0/170990106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150cae0 0x17401d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f120400fea0 tx=0x7f1204014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca8bc894142524d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f11eb7fe700 1 -- 172.21.15.38:0/170990106 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204013ca0 con 0x150cae0 2024-08-15T17:59:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f11eb7fe700 1 -- 172.21.15.38:0/170990106 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120400c040 con 0x150cae0 2024-08-15T17:59:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f11eb7fe700 1 -- 172.21.15.38:0/170990106 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204027680 con 0x150cae0 2024-08-15T17:59:26.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f12128c3740 1 -- 172.21.15.38:0/170990106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150cae0 msgr2=0x17401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f12128c3740 1 --2- 172.21.15.38:0/170990106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150cae0 0x17401d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f120400fea0 tx=0x7f1204014000 comp rx=0 tx=0).stop 2024-08-15T17:59:26.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f12128c3740 1 -- 172.21.15.38:0/170990106 shutdown_connections 2024-08-15T17:59:26.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f12128c3740 1 --2- 172.21.15.38:0/170990106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x1557790 0x1743c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f12128c3740 1 --2- 172.21.15.38:0/170990106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x1740710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f12128c3740 1 --2- 172.21.15.38:0/170990106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150cae0 0x17401d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f12128c3740 1 -- 172.21.15.38:0/170990106 >> 172.21.15.38:0/170990106 conn(0x150ceb0 msgr2=0x17460b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:26.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f12128c3740 1 -- 172.21.15.38:0/170990106 shutdown_connections 2024-08-15T17:59:26.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f12128c3740 1 -- 172.21.15.38:0/170990106 wait complete. 2024-08-15T17:59:26.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f12128c3740 1 -- start start 2024-08-15T17:59:26.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.629+0000 7f12128c3740 1 Processor -- start 2024-08-15T17:59:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150c710 0x178d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150cae0 0x178ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x1557790 0x1748430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x1679d40 con 0x150c710 2024-08-15T17:59:26.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1652970 con 0x150cae0 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x16bb070 con 0x1557790 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f120e3b0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150c710 0x178d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 10 client.0 _ll_register_callbacks cb 0x1656f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f120e3b0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150c710 0x178d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35504/0 (socket says 172.21.15.38:35504) 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f120e3b0700 1 -- 172.21.15.38:0/817520328 learned_addr learned my addr 172.21.15.38:0/817520328 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f120dbaf700 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150cae0 0x178ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f120ebb1700 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x1557790 0x1748430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f120e3b0700 1 -- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x1557790 msgr2=0x1748430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f120e3b0700 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x1557790 0x1748430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f120e3b0700 1 -- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150cae0 msgr2=0x178ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f120e3b0700 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150cae0 0x178ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f120e3b0700 1 -- 172.21.15.38:0/817520328 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120400fad0 con 0x150c710 2024-08-15T17:59:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f120e3b0700 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150c710 0x178d890 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f120400ab70 tx=0x7f1204021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d971d2d619cde364 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f11e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f11e8ff9700 1 -- 172.21.15.38:0/817520328 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120402c070 con 0x150c710 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f11e8ff9700 1 -- 172.21.15.38:0/817520328 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1204031460 con 0x150c710 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 1 -- 172.21.15.38:0/817520328 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x151a260 con 0x150c710 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f11e8ff9700 1 -- 172.21.15.38:0/817520328 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1204031830 con 0x150c710 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 10 client.5184 fetch_fsmap learned FSMap version 15 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 1 -- 172.21.15.38:0/817520328 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x1682370 con 0x150c710 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f11e8ff9700 1 -- 172.21.15.38:0/817520328 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x151a260 con 0x150c710 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f11e8ff9700 1 -- 172.21.15.38:0/817520328 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 15) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1204031990 con 0x150c710 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 10 client.5184 fetch_fsmap finished waiting for FSMap version 15 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 10 client.5184 resolve_mds: resolved name 'd' to daemon mds.d 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 20 client.5184 populate_metadata read hostname 'smithi038' 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:6836/1865762791,v1:172.21.15.104:6837/1865762791] conn(0x1796a20 0x1798e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 4 client.5184 mds_command: new command op to 4999 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-15T17:59:26.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.633+0000 7f12128c3740 1 -- 172.21.15.38:0/817520328 --> [v2:172.21.15.104:6836/1865762791,v1:172.21.15.104:6837/1865762791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x15644e0 con 0x1796a20 2024-08-15T17:59:26.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f120dbaf700 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:6836/1865762791,v1:172.21.15.104:6837/1865762791] conn(0x1796a20 0x1798e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f11e8ff9700 10 client.5184 ms_handle_connect on v2:172.21.15.104:6836/1865762791 2024-08-15T17:59:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f120dbaf700 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:6836/1865762791,v1:172.21.15.104:6837/1865762791] conn(0x1796a20 0x1798e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d3ede5e0024143 server_cookie=8a27f64e6299b773 in_seq=0 out_seq=0 2024-08-15T17:59:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f11e8ff9700 1 -- 172.21.15.38:0/817520328 <== mds.0 v2:172.21.15.104:6836/1865762791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0x15644e0 con 0x1796a20 2024-08-15T17:59:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f11e8ff9700 10 client.5184 handle_command_reply: tid=0 2024-08-15T17:59:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 1 client.5184 shutdown 2024-08-15T17:59:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 1 -- 172.21.15.38:0/817520328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150c710 msgr2=0x178d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150c710 0x178d890 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f120400ab70 tx=0x7f1204021000 comp rx=0 tx=0).stop 2024-08-15T17:59:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 1 -- 172.21.15.38:0/817520328 shutdown_connections 2024-08-15T17:59:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:6836/1865762791,v1:172.21.15.104:6837/1865762791] conn(0x1796a20 0x1798e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x1557790 0x1748430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150cae0 0x178ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f11e8ff9700 0 client.5184 ms_handle_reset on v2:172.21.15.104:6836/1865762791 2024-08-15T17:59:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 1 --2- 172.21.15.38:0/817520328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x150c710 0x178d890 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 1 -- 172.21.15.38:0/817520328 >> 172.21.15.38:0/817520328 conn(0x150ceb0 msgr2=0x1741c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 1 -- 172.21.15.38:0/817520328 shutdown_connections 2024-08-15T17:59:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 1 -- 172.21.15.38:0/817520328 wait complete. 2024-08-15T17:59:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.637+0000 7f12128c3740 20 client.5184 trim_cache size 0 max 16384 2024-08-15T17:59:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f12128c3740 1 Processor -- start 2024-08-15T17:59:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f12128c3740 1 -- start start 2024-08-15T17:59:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f12128c3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x1747000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f12128c3740 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x150cae0 0x1747540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f12128c3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f12128c3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x167e9c0 con 0x1557790 2024-08-15T17:59:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f12128c3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x16b8640 con 0x150c710 2024-08-15T17:59:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f12128c3740 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x162dcb0 con 0x150cae0 2024-08-15T17:59:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120ebb1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120ebb1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35520/0 (socket says 172.21.15.38:35520) 2024-08-15T17:59:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120ebb1700 1 -- 172.21.15.38:0/2707879684 learned_addr learned my addr 172.21.15.38:0/2707879684 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120dbaf700 1 --2- 172.21.15.38:0/2707879684 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x150cae0 0x1747540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120e3b0700 1 --2- 172.21.15.38:0/2707879684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x1747000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120ebb1700 1 -- 172.21.15.38:0/2707879684 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x150cae0 msgr2=0x1747540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120ebb1700 1 --2- 172.21.15.38:0/2707879684 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x150cae0 0x1747540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120ebb1700 1 -- 172.21.15.38:0/2707879684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 msgr2=0x1747000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120ebb1700 1 --2- 172.21.15.38:0/2707879684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x1747000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120ebb1700 1 -- 172.21.15.38:0/2707879684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1668d80 con 0x1557790 2024-08-15T17:59:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120dbaf700 1 --2- 172.21.15.38:0/2707879684 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x150cae0 0x1747540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f120ebb1700 1 --2- 172.21.15.38:0/2707879684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a1c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1200001840 tx=0x7f1200009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3d51421e4febfd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f11ebfff700 1 -- 172.21.15.38:0/2707879684 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200003960 con 0x1557790 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f11ebfff700 1 -- 172.21.15.38:0/2707879684 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1200003ac0 con 0x1557790 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.661+0000 7f11ebfff700 1 -- 172.21.15.38:0/2707879684 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200003e80 con 0x1557790 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 -- 172.21.15.38:0/2707879684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 msgr2=0x17a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 --2- 172.21.15.38:0/2707879684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a1c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1200001840 tx=0x7f1200009040 comp rx=0 tx=0).stop 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 -- 172.21.15.38:0/2707879684 shutdown_connections 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 --2- 172.21.15.38:0/2707879684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a1c80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 --2- 172.21.15.38:0/2707879684 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x150cae0 0x1747540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 --2- 172.21.15.38:0/2707879684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x1747000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 -- 172.21.15.38:0/2707879684 >> 172.21.15.38:0/2707879684 conn(0x150ceb0 msgr2=0x17452a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:26.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 -- 172.21.15.38:0/2707879684 shutdown_connections 2024-08-15T17:59:26.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 -- 172.21.15.38:0/2707879684 wait complete. 2024-08-15T17:59:26.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 -- start start 2024-08-15T17:59:26.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 Processor -- start 2024-08-15T17:59:26.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x17e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.665+0000 7f12128c3740 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x150cae0 0x17e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x16da910 con 0x1557790 2024-08-15T17:59:26.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1656d30 con 0x150c710 2024-08-15T17:59:26.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x1684820 con 0x150cae0 2024-08-15T17:59:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 10 client.0 _ll_register_callbacks cb 0x14d89d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T17:59:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120ebb1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120ebb1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35536/0 (socket says 172.21.15.38:35536) 2024-08-15T17:59:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120ebb1700 1 -- 172.21.15.38:0/3779839504 learned_addr learned my addr 172.21.15.38:0/3779839504 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120e3b0700 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x17e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120dbaf700 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x150cae0 0x17e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120ebb1700 1 -- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x150cae0 msgr2=0x17e9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:26.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120ebb1700 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x150cae0 0x17e9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120ebb1700 1 -- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 msgr2=0x17e97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120ebb1700 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x17e97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120ebb1700 1 -- 172.21.15.38:0/3779839504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120000f040 con 0x1557790 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120ebb1700 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a4400 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1200001f90 tx=0x7f120001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9159483a33c6ef66 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f11e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f11e8ff9700 1 -- 172.21.15.38:0/3779839504 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120001f670 con 0x1557790 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f11e8ff9700 1 -- 172.21.15.38:0/3779839504 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12000278b0 con 0x1557790 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 1 -- 172.21.15.38:0/3779839504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f120400c040 con 0x1557790 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f11e8ff9700 1 -- 172.21.15.38:0/3779839504 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1200027c80 con 0x1557790 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 10 client.5190 fetch_fsmap learned FSMap version 15 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 1 -- 172.21.15.38:0/3779839504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x165d130 con 0x1557790 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f11e8ff9700 1 -- 172.21.15.38:0/3779839504 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120400c040 con 0x1557790 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f11e8ff9700 1 -- 172.21.15.38:0/3779839504 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 15) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f120002a070 con 0x1557790 2024-08-15T17:59:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 10 client.5190 fetch_fsmap finished waiting for FSMap version 15 2024-08-15T17:59:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 10 client.5190 resolve_mds: resolved name 'd' to daemon mds.d 2024-08-15T17:59:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 20 client.5190 populate_metadata read hostname 'smithi038' 2024-08-15T17:59:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:6836/1865762791,v1:172.21.15.104:6837/1865762791] conn(0x17a4dc0 0x17a5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 4 client.5190 mds_command: new command op to 4999 tid=0[{"prefix": "status"}] 2024-08-15T17:59:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f12128c3740 1 -- 172.21.15.38:0/3779839504 --> [v2:172.21.15.104:6836/1865762791,v1:172.21.15.104:6837/1865762791] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1589260 con 0x17a4dc0 2024-08-15T17:59:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.669+0000 7f120e3b0700 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:6836/1865762791,v1:172.21.15.104:6837/1865762791] conn(0x17a4dc0 0x17a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f11e8ff9700 10 client.5190 ms_handle_connect on v2:172.21.15.104:6836/1865762791 2024-08-15T17:59:26.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120e3b0700 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:6836/1865762791,v1:172.21.15.104:6837/1865762791] conn(0x17a4dc0 0x17a5190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2339c436be1b42af server_cookie=30ac97dc639a7077 in_seq=0 out_seq=0 2024-08-15T17:59:26.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f11e8ff9700 1 -- 172.21.15.38:0/3779839504 <== mds.0 v2:172.21.15.104:6836/1865762791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x1589260 con 0x17a4dc0 2024-08-15T17:59:26.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f11e8ff9700 10 client.5190 handle_command_reply: tid=0 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 1 client.5190 shutdown 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 1 -- 172.21.15.38:0/3779839504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 msgr2=0x17a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a4400 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1200001f90 tx=0x7f120001d000 comp rx=0 tx=0).stop 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 1 -- 172.21.15.38:0/3779839504 shutdown_connections 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:6836/1865762791,v1:172.21.15.104:6837/1865762791] conn(0x17a4dc0 0x17a5190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x1557790 0x17a4400 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f11e8ff9700 0 client.5190 ms_handle_reset on v2:172.21.15.104:6836/1865762791 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x150cae0 0x17e9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 1 --2- 172.21.15.38:0/3779839504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x150c710 0x17e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 1 -- 172.21.15.38:0/3779839504 >> 172.21.15.38:0/3779839504 conn(0x150ceb0 msgr2=0x1747890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 1 -- 172.21.15.38:0/3779839504 shutdown_connections 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 1 -- 172.21.15.38:0/3779839504 wait complete. 2024-08-15T17:59:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f12128c3740 20 client.5190 trim_cache size 0 max 16384 2024-08-15T17:59:26.675 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T17:59:26.675 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "3c800201-a1b3-4c6d-887f-37b16c41eb0e", 2024-08-15T17:59:26.675 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-08-15T17:59:26.675 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4999, 2024-08-15T17:59:26.675 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-08-15T17:59:26.675 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-08-15T17:59:26.675 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-08-15T17:59:26.675 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 5.677236121, 2024-08-15T17:59:26.675 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 15, 2024-08-15T17:59:26.675 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 32, 2024-08-15T17:59:26.676 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 32, 2024-08-15T17:59:26.676 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 12.762468576 2024-08-15T17:59:26.676 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T17:59:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 -- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f11ec060860 msgr2=0x7f11ec062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f11ec060860 0x7f11ec062d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f120814f870 tx=0x7f1204006f90 comp rx=0 tx=0).stop 2024-08-15T17:59:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 -- 172.21.15.38:0/4059572751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120806e9d0 msgr2=0x7f1208147b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120806e9d0 0x7f1208147b60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009290 tx=0x7f11fc017000 comp rx=0 tx=0).stop 2024-08-15T17:59:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 -- 172.21.15.38:0/4059572751 shutdown_connections 2024-08-15T17:59:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f11ec060860 0x7f11ec062d10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12080ab540 0x7f120814b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120806e9d0 0x7f1208147b60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 --2- 172.21.15.38:0/4059572751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f120806df10 0x7f1208135620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 -- 172.21.15.38:0/4059572751 >> 172.21.15.38:0/4059572751 conn(0x7f12080678d0 msgr2=0x7f12081542b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:26.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 -- 172.21.15.38:0/4059572751 shutdown_connections 2024-08-15T17:59:26.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.673+0000 7f120f3b2700 1 -- 172.21.15.38:0/4059572751 wait complete. 2024-08-15T17:59:26.690 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:59:26.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.801+0000 7fb52ffff700 1 Processor -- start 2024-08-15T17:59:26.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52ffff700 1 -- start start 2024-08-15T17:59:26.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb530159040 0x7fb5300cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52ffff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5300cf190 con 0x7fb5300d0010 2024-08-15T17:59:26.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52ffff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5300ce5e0 con 0x7fb53015d5b0 2024-08-15T17:59:26.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52ffff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53015d9f0 con 0x7fb530159040 2024-08-15T17:59:26.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52e7fc700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb5300d0010 0x7fb53015cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:46296/0 (socket says 172.21.15.38:46296) 2024-08-15T17:59:26.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52e7fc700 1 -- 172.21.15.38:0/2792257046 learned_addr learned my addr 172.21.15.38:0/2792257046 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52effd700 1 --2- 172.21.15.38:0/2792257046 >> v2:172.21.15.104:3300/0 conn(0x7fb530159040 0x7fb5300cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52dffb700 1 -- 172.21.15.38:0/2792257046 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547028468 0 0) 0x7fb5300cf190 con 0x7fb5300d0010 2024-08-15T17:59:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52effd700 1 -- 172.21.15.38:0/2792257046 >> v1:172.21.15.104:6789/0 conn(0x7fb53015d5b0 legacy=0x7fb53015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52effd700 1 -- 172.21.15.38:0/2792257046 >> v1:172.21.15.38:6789/0 conn(0x7fb5300d0010 legacy=0x7fb53015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52effd700 1 -- 172.21.15.38:0/2792257046 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5300ce860 con 0x7fb530159040 2024-08-15T17:59:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52effd700 1 --2- 172.21.15.38:0/2792257046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb530159040 0x7fb5300cf620 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb53015db30 tx=0x7fb5200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f395f288247eef2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52dffb700 1 -- 172.21.15.38:0/2792257046 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000b6b0 con 0x7fb530159040 2024-08-15T17:59:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52dffb700 1 -- 172.21.15.38:0/2792257046 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52000b810 con 0x7fb530159040 2024-08-15T17:59:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.805+0000 7fb52dffb700 1 -- 172.21.15.38:0/2792257046 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000bbd0 con 0x7fb530159040 2024-08-15T17:59:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 -- 172.21.15.38:0/2792257046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb530159040 msgr2=0x7fb5300cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 --2- 172.21.15.38:0/2792257046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb530159040 0x7fb5300cf620 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb53015db30 tx=0x7fb5200091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 -- 172.21.15.38:0/2792257046 shutdown_connections 2024-08-15T17:59:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 --2- 172.21.15.38:0/2792257046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb530159040 0x7fb5300cf620 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 -- 172.21.15.38:0/2792257046 >> 172.21.15.38:0/2792257046 conn(0x7fb530068250 msgr2=0x7fb530068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 -- 172.21.15.38:0/2792257046 shutdown_connections 2024-08-15T17:59:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 -- 172.21.15.38:0/2792257046 wait complete. 2024-08-15T17:59:26.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 Processor -- start 2024-08-15T17:59:26.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 -- start start 2024-08-15T17:59:26.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5300d0010 0x7fb5300b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb530159040 0x7fb53016cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53015d5b0 0x7fb5300cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5301526c0 con 0x7fb5300d0010 2024-08-15T17:59:26.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb530169d50 con 0x7fb53015d5b0 2024-08-15T17:59:26.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52ffff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53016d4f0 con 0x7fb530159040 2024-08-15T17:59:26.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5300d0010 0x7fb5300b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5300d0010 0x7fb5300b2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35542/0 (socket says 172.21.15.38:35542) 2024-08-15T17:59:26.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52effd700 1 -- 172.21.15.38:0/3350293283 learned_addr learned my addr 172.21.15.38:0/3350293283 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:26.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52f7fe700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53015d5b0 0x7fb5300cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52e7fc700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb530159040 0x7fb53016cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52f7fe700 1 -- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb530159040 msgr2=0x7fb53016cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52f7fe700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb530159040 0x7fb53016cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52f7fe700 1 -- 172.21.15.38:0/3350293283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5300d0010 msgr2=0x7fb5300b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:26.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52f7fe700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5300d0010 0x7fb5300b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:26.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.809+0000 7fb52f7fe700 1 -- 172.21.15.38:0/3350293283 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520009420 con 0x7fb53015d5b0 2024-08-15T17:59:26.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.813+0000 7fb52f7fe700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53015d5b0 0x7fb5300cc620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb52800c830 tx=0x7fb528001690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:26.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.813+0000 7fb526ffd700 1 -- 172.21.15.38:0/3350293283 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528012630 con 0x7fb53015d5b0 2024-08-15T17:59:26.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.813+0000 7fb526ffd700 1 -- 172.21.15.38:0/3350293283 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb528012790 con 0x7fb53015d5b0 2024-08-15T17:59:26.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.813+0000 7fb526ffd700 1 -- 172.21.15.38:0/3350293283 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528012b50 con 0x7fb53015d5b0 2024-08-15T17:59:26.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.813+0000 7fb52ffff700 1 -- 172.21.15.38:0/3350293283 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53016d790 con 0x7fb53015d5b0 2024-08-15T17:59:26.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.813+0000 7fb52ffff700 1 -- 172.21.15.38:0/3350293283 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5301699b0 con 0x7fb53015d5b0 2024-08-15T17:59:26.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.813+0000 7fb52ffff700 1 -- 172.21.15.38:0/3350293283 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb530169f00 con 0x7fb53015d5b0 2024-08-15T17:59:26.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.813+0000 7fb526ffd700 1 -- 172.21.15.38:0/3350293283 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb52801b400 con 0x7fb53015d5b0 2024-08-15T17:59:26.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.813+0000 7fb526ffd700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb510060860 0x7fb510062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:26.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.813+0000 7fb526ffd700 1 -- 172.21.15.38:0/3350293283 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb52807d080 con 0x7fb53015d5b0 2024-08-15T17:59:26.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.817+0000 7fb52effd700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb510060860 0x7fb510062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:26.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.817+0000 7fb52effd700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb510060860 0x7fb510062d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb5200093f0 tx=0x7fb520016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:26.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:26.825+0000 7fb526ffd700 1 -- 172.21.15.38:0/3350293283 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb52804d4d0 con 0x7fb53015d5b0 2024-08-15T17:59:27.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.029+0000 7fb52ffff700 1 -- 172.21.15.38:0/3350293283 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb53016db90 con 0x7fb53015d5b0 2024-08-15T17:59:27.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.033+0000 7fb526ffd700 1 -- 172.21.15.38:0/3350293283 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb528050af0 con 0x7fb53015d5b0 2024-08-15T17:59:27.033 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:59:27.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 -- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb510060860 msgr2=0x7fb510062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:27.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb510060860 0x7fb510062d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb5200093f0 tx=0x7fb520016000 comp rx=0 tx=0).stop 2024-08-15T17:59:27.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 -- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53015d5b0 msgr2=0x7fb5300cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:27.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53015d5b0 0x7fb5300cc620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb52800c830 tx=0x7fb528001690 comp rx=0 tx=0).stop 2024-08-15T17:59:27.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 -- 172.21.15.38:0/3350293283 shutdown_connections 2024-08-15T17:59:27.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb510060860 0x7fb510062d10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:27.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb53015d5b0 0x7fb5300cc620 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:27.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb530159040 0x7fb53016cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:27.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 --2- 172.21.15.38:0/3350293283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5300d0010 0x7fb5300b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:27.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 -- 172.21.15.38:0/3350293283 >> 172.21.15.38:0/3350293283 conn(0x7fb530068250 msgr2=0x7fb5300b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:27.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 -- 172.21.15.38:0/3350293283 shutdown_connections 2024-08-15T17:59:27.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:27.037+0000 7fb52ffff700 1 -- 172.21.15.38:0/3350293283 wait complete. 2024-08-15T17:59:27.048 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:59:27.048 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:59:27.048 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-08-15T17:59:27.048 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-15T17:59:27.048 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-15T17:59:27.048 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:59:27.048 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:59:27.048 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:59:27.048 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:59:27.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:27.048 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-08-15T17:59:27.053 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:59:27.053 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42579sec preferred_lft 42579sec 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:27.054 INFO:teuthology.orchestra.run.smithi038.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 02:42:40:33:56:ee brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether b6:52:e5:4f:e7:cd brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::2cd6:e8ff:fe2f:5f2a/64 scope link 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:27.055 INFO:teuthology.orchestra.run.smithi038.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:59:27.056 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether b6:52:e5:4f:e7:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T17:59:27.056 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::b452:e5ff:fe4f:e7cd/64 scope link 2024-08-15T17:59:27.056 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:27.056 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:59:27.056 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ca:f0:f1:ce:60:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-15T17:59:27.056 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::c8f0:f1ff:fece:60d8/64 scope link 2024-08-15T17:59:27.056 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:27.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:27.056 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:59:27.069 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-08-15T17:59:27.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:27.070 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-08-15T17:59:27.123 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-08-15T17:59:27.123 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:59:27.123 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-08-15T17:59:27.123 INFO:teuthology.orchestra.run.smithi038.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:59:27.123 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-08-15T17:59:27.123 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:59:27.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:27.124 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-08-15T17:59:27.124 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-08-15T17:59:27.124 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-15T17:59:27.124 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-15T17:59:27.124 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-15T17:59:27.124 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-08-15T17:59:27.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:27.162 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-08-15T17:59:27.168 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-08-15T17:59:27.168 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-15T17:59:27.168 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:59:27.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:27.168 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-08-15T17:59:27.180 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-08-15T17:59:27.180 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:59:27.180 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-15T17:59:27.229 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:59:27.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:59:27.229 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-15T17:59:27.235 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-08-15T17:59:27.253 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:59:27.253 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:59:27.263 INFO:teuthology.orchestra.run.smithi038.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:59:27.264 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:59:27.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:27.264 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-08-15T17:59:27.269 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T17:59:27.269 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-08-15T17:59:27.312 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-08-15T17:59:27.329 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:59:27.329 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:59:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:59:27.343 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:59:27.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:27.344 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-08-15T17:59:27.402 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-08-15T17:59:27.401+0000 7fc296935380 -1 init, newargv = 0x5595a21e6ad0 newargc=15 2024-08-15T17:59:27.402 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[33069]: starting ceph client 2024-08-15T17:59:27.411 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[33069]: starting fuse 2024-08-15T17:59:28.451 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-08-15T17:59:28.470 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:59:28.470 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:59:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:59:28.528 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:59:28.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:28.529 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-08-15T17:59:28.534 INFO:teuthology.orchestra.run.smithi038.stdout:52 2024-08-15T17:59:28.534 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-08-15T17:59:28.535 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-08-15T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-08-15T17:59:28.622 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.33069.asok 2024-08-15T17:59:28.806 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33069.asok 2024-08-15T17:59:28.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:28.806 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33069.asok status 2024-08-15T17:59:28.949 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T17:59:28.949 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-08-15T17:59:28.949 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:59:28.949 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:59:28.949 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-08-15T17:59:28.950 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-08-15T17:59:28.950 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-08-15T17:59:28.950 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "33069", 2024-08-15T17:59:28.950 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-08-15T17:59:28.950 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:28.950 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-08-15T17:59:28.950 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-08-15T17:59:28.950 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 5202, 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 5202 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1275007640 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-08-15T17:59:28.952 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-08-15T17:59:28.952 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:59:28.952 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1275007640 2024-08-15T17:59:28.952 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:28.952 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.5202 192.168.0.1:0/1275007640", 2024-08-15T17:59:28.952 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/1275007640", 2024-08-15T17:59:28.952 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-08-15T17:59:28.952 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 15, 2024-08-15T17:59:28.952 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 32, 2024-08-15T17:59:28.952 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:59:28.952 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-08-15T17:59:28.953 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-08-15T17:59:28.953 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T17:59:28.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:28.959 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-08-15T17:59:28.959 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-08-15T17:59:28.966 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-08-15T17:59:28.966 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-08-15T17:59:28.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:28.967 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-15T17:59:28.985 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:59:29.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c45ec700 1 Processor -- start 2024-08-15T17:59:29.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c45ec700 1 -- start start 2024-08-15T17:59:29.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c45ec700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89bc156f80 0x7f89bc0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:29.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c45ec700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f89bc0ace80 0x7f89bc157350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:29.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c45ec700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89bc0adf50 con 0x7f89bc157890 2024-08-15T17:59:29.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c45ec700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89bc0ae920 con 0x7f89bc0ace80 2024-08-15T17:59:29.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c45ec700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc0ad340 con 0x7f89bc156f80 2024-08-15T17:59:29.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c1b87700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f89bc0ace80 0x7f89bc157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:29.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c2b89700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f89bc157890 0x7f89bc0aab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:46308/0 (socket says 172.21.15.38:46308) 2024-08-15T17:59:29.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c2b89700 1 -- 172.21.15.38:0/1558548234 learned_addr learned my addr 172.21.15.38:0/1558548234 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:29.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c1b87700 1 --2- 172.21.15.38:0/1558548234 >> v2:172.21.15.38:3300/0 conn(0x7f89bc0ace80 0x7f89bc157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35546/0 (socket says 172.21.15.38:35546) 2024-08-15T17:59:29.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c2388700 1 --2- 172.21.15.38:0/1558548234 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89bc156f80 0x7f89bc0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:29.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c1b87700 1 -- 172.21.15.38:0/1558548234 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89bc156f80 msgr2=0x7f89bc0ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:29.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c1b87700 1 --2- 172.21.15.38:0/1558548234 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89bc156f80 0x7f89bc0ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:29.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c1b87700 1 -- 172.21.15.38:0/1558548234 >> v1:172.21.15.38:6789/0 conn(0x7f89bc157890 legacy=0x7f89bc0aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:29.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c1b87700 1 -- 172.21.15.38:0/1558548234 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc0ad620 con 0x7f89bc0ace80 2024-08-15T17:59:29.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c1386700 1 -- 172.21.15.38:0/1558548234 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600365324 0 0) 0x7f89bc0adf50 con 0x7f89bc157890 2024-08-15T17:59:29.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c2388700 1 --2- 172.21.15.38:0/1558548234 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89bc156f80 0x7f89bc0ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:29.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c1b87700 1 --2- 172.21.15.38:0/1558548234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89bc0ace80 0x7f89bc157350 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009e60 tx=0x7f89b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7900b67d5798bd24 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:29.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c1386700 1 -- 172.21.15.38:0/1558548234 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b800e070 con 0x7f89bc0ace80 2024-08-15T17:59:29.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c1386700 1 -- 172.21.15.38:0/1558548234 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b800bc30 con 0x7f89bc0ace80 2024-08-15T17:59:29.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.141+0000 7f89c1386700 1 -- 172.21.15.38:0/1558548234 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8012430 con 0x7f89bc0ace80 2024-08-15T17:59:29.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 -- 172.21.15.38:0/1558548234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89bc0ace80 msgr2=0x7f89bc157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:29.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 --2- 172.21.15.38:0/1558548234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89bc0ace80 0x7f89bc157350 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009e60 tx=0x7f89b8009780 comp rx=0 tx=0).stop 2024-08-15T17:59:29.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 -- 172.21.15.38:0/1558548234 shutdown_connections 2024-08-15T17:59:29.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 --2- 172.21.15.38:0/1558548234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89bc0ace80 0x7f89bc157350 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:29.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 --2- 172.21.15.38:0/1558548234 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89bc156f80 0x7f89bc0ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:29.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 -- 172.21.15.38:0/1558548234 >> 172.21.15.38:0/1558548234 conn(0x7f89bc066e00 msgr2=0x7f89bc067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:29.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 -- 172.21.15.38:0/1558548234 shutdown_connections 2024-08-15T17:59:29.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 -- 172.21.15.38:0/1558548234 wait complete. 2024-08-15T17:59:29.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 Processor -- start 2024-08-15T17:59:29.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 -- start start 2024-08-15T17:59:29.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89bc0ace80 0x7f89bc0c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:29.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89bc156f80 0x7f89bc098ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:29.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89bc157890 0x7f89bc153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:29.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc169450 con 0x7f89bc0ace80 2024-08-15T17:59:29.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc0b0360 con 0x7f89bc157890 2024-08-15T17:59:29.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c45ec700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc0b0640 con 0x7f89bc156f80 2024-08-15T17:59:29.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.145+0000 7f89c2388700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89bc0ace80 0x7f89bc0c0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:29.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c2388700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89bc0ace80 0x7f89bc0c0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35558/0 (socket says 172.21.15.38:35558) 2024-08-15T17:59:29.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c2388700 1 -- 172.21.15.38:0/2092589910 learned_addr learned my addr 172.21.15.38:0/2092589910 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:29.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c2b89700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89bc157890 0x7f89bc153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:29.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c1b87700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89bc156f80 0x7f89bc098ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:29.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c2b89700 1 -- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89bc156f80 msgr2=0x7f89bc098ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:29.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c2b89700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89bc156f80 0x7f89bc098ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:29.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c2b89700 1 -- 172.21.15.38:0/2092589910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89bc0ace80 msgr2=0x7f89bc0c0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:29.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c2b89700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89bc0ace80 0x7f89bc0c0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:29.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c2b89700 1 -- 172.21.15.38:0/2092589910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b8009a20 con 0x7f89bc157890 2024-08-15T17:59:29.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c2b89700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89bc157890 0x7f89bc153040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f89b400c440 tx=0x7f89b4007ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:29.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89b27fc700 1 -- 172.21.15.38:0/2092589910 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4005900 con 0x7f89bc157890 2024-08-15T17:59:29.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89b27fc700 1 -- 172.21.15.38:0/2092589910 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b4005a80 con 0x7f89bc157890 2024-08-15T17:59:29.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c45ec700 1 -- 172.21.15.38:0/2092589910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89bc0b0920 con 0x7f89bc157890 2024-08-15T17:59:29.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89b27fc700 1 -- 172.21.15.38:0/2092589910 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4005e40 con 0x7f89bc157890 2024-08-15T17:59:29.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c45ec700 1 -- 172.21.15.38:0/2092589910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89bc0c13e0 con 0x7f89bc157890 2024-08-15T17:59:29.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89c45ec700 1 -- 172.21.15.38:0/2092589910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89bc0645c0 con 0x7f89bc157890 2024-08-15T17:59:29.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.149+0000 7f89b27fc700 1 -- 172.21.15.38:0/2092589910 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f89b40072e0 con 0x7f89bc157890 2024-08-15T17:59:29.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.153+0000 7f89b27fc700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89a0060860 0x7f89a0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:29.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.153+0000 7f89b27fc700 1 -- 172.21.15.38:0/2092589910 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f89b407bef0 con 0x7f89bc157890 2024-08-15T17:59:29.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.153+0000 7f89c2388700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89a0060860 0x7f89a0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:29.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.153+0000 7f89c2388700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89a0060860 0x7f89a0062d10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f89ac006dc0 tx=0x7f89ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:29.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.165+0000 7f89b27fc700 1 -- 172.21.15.38:0/2092589910 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f89b404c2a0 con 0x7f89bc157890 2024-08-15T17:59:29.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.365+0000 7f89c45ec700 1 -- 172.21.15.38:0/2092589910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f89bc0b0da0 con 0x7f89bc157890 2024-08-15T17:59:29.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.365+0000 7f89b27fc700 1 -- 172.21.15.38:0/2092589910 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f89b404f8c0 con 0x7f89bc157890 2024-08-15T17:59:29.366 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:59:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 -- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89a0060860 msgr2=0x7f89a0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89a0060860 0x7f89a0062d10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f89ac006dc0 tx=0x7f89ac008040 comp rx=0 tx=0).stop 2024-08-15T17:59:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 -- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89bc157890 msgr2=0x7f89bc153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89bc157890 0x7f89bc153040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f89b400c440 tx=0x7f89b4007ed0 comp rx=0 tx=0).stop 2024-08-15T17:59:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 -- 172.21.15.38:0/2092589910 shutdown_connections 2024-08-15T17:59:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f89a0060860 0x7f89a0062d10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89bc157890 0x7f89bc153040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89bc156f80 0x7f89bc098ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:29.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 --2- 172.21.15.38:0/2092589910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89bc0ace80 0x7f89bc0c0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:29.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 -- 172.21.15.38:0/2092589910 >> 172.21.15.38:0/2092589910 conn(0x7f89bc066e00 msgr2=0x7f89bc06d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:29.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 -- 172.21.15.38:0/2092589910 shutdown_connections 2024-08-15T17:59:29.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:29.369+0000 7f89c45ec700 1 -- 172.21.15.38:0/2092589910 wait complete. 2024-08-15T17:59:29.387 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:59:29.387 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:59:29.387 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-08-15T17:59:29.387 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-08-15T17:59:29.387 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-08-15T17:59:29.387 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:59:29.388 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:59:29.388 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:59:29.388 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:59:29.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:29.388 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-08-15T17:59:29.396 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:59:29.396 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:59:29.396 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:59:29.397 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42568sec preferred_lft 42568sec 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:51:73:f0:c7 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 2a:69:3f:88:03:6f brd ff:ff:ff:ff:ff:ff 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::7049:8aff:fe98:12d7/64 scope link 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:29.398 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:59:29.399 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 2a:69:3f:88:03:6f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T17:59:29.399 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::2869:3fff:fe88:36f/64 scope link 2024-08-15T17:59:29.399 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:29.399 INFO:teuthology.orchestra.run.smithi104.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:59:29.399 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 76:19:b9:ab:00:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-15T17:59:29.399 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::7419:b9ff:feab:f7/64 scope link 2024-08-15T17:59:29.399 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:59:29.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:29.399 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:59:29.452 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-08-15T17:59:29.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:29.454 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-08-15T17:59:29.464 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-08-15T17:59:29.464 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:59:29.464 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:59:29.464 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:59:29.464 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:59:29.464 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:59:29.464 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:59:29.464 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:59:29.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:29.464 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2024-08-15T17:59:29.465 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-08-15T17:59:29.465 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:59:29.465 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:59:29.465 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:59:29.465 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-08-15T17:59:29.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:29.502 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-08-15T17:59:29.552 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-08-15T17:59:29.552 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-15T17:59:29.552 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:59:29.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:59:29.553 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-08-15T17:59:29.568 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-08-15T17:59:29.568 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:59:29.568 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-08-15T17:59:29.574 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:59:29.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:59:29.575 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:59:29.581 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2024-08-15T17:59:29.639 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:59:29.639 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:59:29.655 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:59:29.655 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:59:29.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:29.656 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2024-08-15T17:59:29.661 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T17:59:29.662 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-08-15T17:59:29.666 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2024-08-15T17:59:29.683 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:59:29.683 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:59:29.738 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:59:29.739 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:59:29.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:29.739 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2024-08-15T17:59:29.750 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi104.stderr:2024-08-15T17:59:29.750+0000 7f662a0d2380 -1 init, newargv = 0x555960760330 newargc=15 2024-08-15T17:59:29.750 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi104.stderr:ceph-fuse[30232]: starting ceph client 2024-08-15T17:59:29.758 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi104.stderr:ceph-fuse[30232]: starting fuse 2024-08-15T17:59:30.847 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2024-08-15T17:59:30.866 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:59:30.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:59:30.922 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:59:30.922 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:59:30.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:30.923 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2024-08-15T17:59:30.928 INFO:teuthology.orchestra.run.smithi104.stdout:52 2024-08-15T17:59:30.928 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-15T17:59:30.928 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:59:30.928 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2024-08-15T17:59:30.928 DEBUG:teuthology.orchestra.run.smithi104:> import re 2024-08-15T17:59:30.928 DEBUG:teuthology.orchestra.run.smithi104:> import os 2024-08-15T17:59:30.928 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2024-08-15T17:59:30.928 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:59:30.928 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2024-08-15T17:59:30.928 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2024-08-15T17:59:30.929 DEBUG:teuthology.orchestra.run.smithi104:> return f 2024-08-15T17:59:30.930 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:59:30.930 DEBUG:teuthology.orchestra.run.smithi104:> 2024-08-15T17:59:30.930 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.1")) 2024-08-15T17:59:30.930 DEBUG:teuthology.orchestra.run.smithi104:> ' 2024-08-15T17:59:31.009 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.1.30232.asok 2024-08-15T17:59:31.195 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.30232.asok 2024-08-15T17:59:31.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:31.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.30232.asok status 2024-08-15T17:59:31.285 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-08-15T17:59:31.285 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2024-08-15T17:59:31.285 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:59:31.285 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:59:31.285 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "1", 2024-08-15T17:59:31.286 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2024-08-15T17:59:31.286 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-08-15T17:59:31.286 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "30232", 2024-08-15T17:59:31.286 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2024-08-15T17:59:31.286 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:59:31.286 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2024-08-15T17:59:31.286 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2024-08-15T17:59:31.286 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 4895, 2024-08-15T17:59:31.286 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2024-08-15T17:59:31.286 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2024-08-15T17:59:31.286 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 4895 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1756234007 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1756234007 2024-08-15T17:59:31.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-08-15T17:59:31.288 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.4895 192.168.0.1:0/1756234007", 2024-08-15T17:59:31.288 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "192.168.0.1:0/1756234007", 2024-08-15T17:59:31.288 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2024-08-15T17:59:31.288 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 15, 2024-08-15T17:59:31.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 32, 2024-08-15T17:59:31.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:59:31.288 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2024-08-15T17:59:31.288 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2024-08-15T17:59:31.288 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-08-15T17:59:31.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:31.295 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2024-08-15T17:59:31.295 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.1 2024-08-15T17:59:31.300 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2024-08-15T17:59:31.301 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-08-15T17:59:31.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:59:31.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:59:31.311 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-08-15T17:59:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.437+0000 7f209d068700 1 Processor -- start 2024-08-15T17:59:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.437+0000 7f209d068700 1 -- start start 2024-08-15T17:59:31.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f20980cd2b0 0x7f20980cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:31.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20980caeb0 0x7f20980ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:31.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f20980cb300 0x7f20980ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:31.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20980cc480 con 0x7f20980cb300 2024-08-15T17:59:31.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2098159720 con 0x7f20980cd2b0 2024-08-15T17:59:31.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20980cb870 con 0x7f20980caeb0 2024-08-15T17:59:31.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209759e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f20980cb300 0x7f20980ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209759e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f20980cb300 0x7f20980ce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35562/0 (socket says 172.21.15.38:35562) 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209759e700 1 -- 172.21.15.38:0/4052917981 learned_addr learned my addr 172.21.15.38:0/4052917981 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209659c700 1 --2- 172.21.15.38:0/4052917981 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20980caeb0 0x7f20980ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f2096d9d700 1 --2- 172.21.15.38:0/4052917981 >> v2:172.21.15.104:3300/0 conn(0x7f20980cd2b0 0x7f20980cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209759e700 1 -- 172.21.15.38:0/4052917981 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20980caeb0 msgr2=0x7f20980ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209759e700 1 --2- 172.21.15.38:0/4052917981 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20980caeb0 0x7f20980ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209759e700 1 -- 172.21.15.38:0/4052917981 >> v2:172.21.15.104:3300/0 conn(0x7f20980cd2b0 msgr2=0x7f20980cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209759e700 1 --2- 172.21.15.38:0/4052917981 >> v2:172.21.15.104:3300/0 conn(0x7f20980cd2b0 0x7f20980cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209759e700 1 -- 172.21.15.38:0/4052917981 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20980cbb50 con 0x7f20980cb300 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209759e700 1 --2- 172.21.15.38:0/4052917981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20980cb300 0x7f20980ce9a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f209808a610 tx=0x7f2090009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4230a20a04fb3f62 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f2095d9b700 1 -- 172.21.15.38:0/4052917981 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209000e070 con 0x7f20980cb300 2024-08-15T17:59:31.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f2095d9b700 1 -- 172.21.15.38:0/4052917981 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209000b970 con 0x7f20980cb300 2024-08-15T17:59:31.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f2095d9b700 1 -- 172.21.15.38:0/4052917981 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900092e0 con 0x7f20980cb300 2024-08-15T17:59:31.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 -- 172.21.15.38:0/4052917981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20980cb300 msgr2=0x7f20980ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:31.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 --2- 172.21.15.38:0/4052917981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20980cb300 0x7f20980ce9a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f209808a610 tx=0x7f2090009840 comp rx=0 tx=0).stop 2024-08-15T17:59:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 -- 172.21.15.38:0/4052917981 shutdown_connections 2024-08-15T17:59:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 --2- 172.21.15.38:0/4052917981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20980cb300 0x7f20980ce9a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 --2- 172.21.15.38:0/4052917981 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20980caeb0 0x7f20980ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 --2- 172.21.15.38:0/4052917981 >> v2:172.21.15.104:3300/0 conn(0x7f20980cd2b0 0x7f20980cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 -- 172.21.15.38:0/4052917981 >> 172.21.15.38:0/4052917981 conn(0x7f2098099550 msgr2=0x7f2098067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 -- 172.21.15.38:0/4052917981 shutdown_connections 2024-08-15T17:59:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.441+0000 7f209d068700 1 -- 172.21.15.38:0/4052917981 wait complete. 2024-08-15T17:59:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.445+0000 7f209d068700 1 Processor -- start 2024-08-15T17:59:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.445+0000 7f209d068700 1 -- start start 2024-08-15T17:59:31.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.445+0000 7f209d068700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20980caeb0 0x7f209814f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:31.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.445+0000 7f209d068700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20980cb300 0x7f20980b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:31.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.445+0000 7f209d068700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20980cd2b0 0x7f20980b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:31.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.445+0000 7f209d068700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20980c7d30 con 0x7f20980caeb0 2024-08-15T17:59:31.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.445+0000 7f209d068700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20980aeff0 con 0x7f20980cd2b0 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.445+0000 7f209d068700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20980c7400 con 0x7f20980cb300 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.445+0000 7f2096d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20980caeb0 0x7f209814f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.445+0000 7f2096d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20980caeb0 0x7f209814f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35566/0 (socket says 172.21.15.38:35566) 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.445+0000 7f2096d9d700 1 -- 172.21.15.38:0/3857492003 learned_addr learned my addr 172.21.15.38:0/3857492003 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f209759e700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20980cd2b0 0x7f20980b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f209659c700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20980cb300 0x7f20980b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f2096d9d700 1 -- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20980cb300 msgr2=0x7f20980b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f2096d9d700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20980cb300 0x7f20980b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f2096d9d700 1 -- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20980cd2b0 msgr2=0x7f20980b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f2096d9d700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20980cd2b0 0x7f20980b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f2096d9d700 1 -- 172.21.15.38:0/3857492003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2090009ae0 con 0x7f20980caeb0 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f209759e700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20980cd2b0 0x7f20980b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f2096d9d700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20980caeb0 0x7f209814f940 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e930 tx=0x7f208c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:31.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f207effd700 1 -- 172.21.15.38:0/3857492003 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c013680 con 0x7f20980caeb0 2024-08-15T17:59:31.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f207effd700 1 -- 172.21.15.38:0/3857492003 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c0137e0 con 0x7f20980caeb0 2024-08-15T17:59:31.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f209d068700 1 -- 172.21.15.38:0/3857492003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20980c76e0 con 0x7f20980caeb0 2024-08-15T17:59:31.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f207effd700 1 -- 172.21.15.38:0/3857492003 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c013b80 con 0x7f20980caeb0 2024-08-15T17:59:31.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f209d068700 1 -- 172.21.15.38:0/3857492003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209814fe80 con 0x7f20980caeb0 2024-08-15T17:59:31.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f209d068700 1 -- 172.21.15.38:0/3857492003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20980638f0 con 0x7f20980caeb0 2024-08-15T17:59:31.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.449+0000 7f207effd700 1 -- 172.21.15.38:0/3857492003 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f208c00c460 con 0x7f20980caeb0 2024-08-15T17:59:31.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.453+0000 7f207effd700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2084060860 0x7f2084062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:31.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.453+0000 7f207effd700 1 -- 172.21.15.38:0/3857492003 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f208c07cdb0 con 0x7f20980caeb0 2024-08-15T17:59:31.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.453+0000 7f209659c700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2084060860 0x7f2084062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:31.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.453+0000 7f209659c700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2084060860 0x7f2084062d10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f209809e110 tx=0x7f20880091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:31.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.457+0000 7f207effd700 1 -- 172.21.15.38:0/3857492003 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f208c081070 con 0x7f20980caeb0 2024-08-15T17:59:31.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:31.661+0000 7f209d068700 1 -- 172.21.15.38:0/3857492003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2098068020 con 0x7f20980caeb0 2024-08-15T17:59:32.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.077+0000 7f207effd700 1 -- 172.21.15.38:0/3857492003 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 136+0+0 (secure 0 0 0) 0x7f208c04d160 con 0x7f20980caeb0 2024-08-15T17:59:32.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 -- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2084060860 msgr2=0x7f2084062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2084060860 0x7f2084062d10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f209809e110 tx=0x7f20880091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 -- 172.21.15.38:0/3857492003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20980caeb0 msgr2=0x7f209814f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20980caeb0 0x7f209814f940 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e930 tx=0x7f208c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 -- 172.21.15.38:0/3857492003 shutdown_connections 2024-08-15T17:59:32.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2084060860 0x7f2084062d10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20980cd2b0 0x7f20980b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20980cb300 0x7f20980b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 --2- 172.21.15.38:0/3857492003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20980caeb0 0x7f209814f940 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 -- 172.21.15.38:0/3857492003 >> 172.21.15.38:0/3857492003 conn(0x7f2098099550 msgr2=0x7f20980c5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:32.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 -- 172.21.15.38:0/3857492003 shutdown_connections 2024-08-15T17:59:32.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.081+0000 7f209d068700 1 -- 172.21.15.38:0/3857492003 wait complete. 2024-08-15T17:59:32.096 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-08-15T17:59:32.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8bf59e700 1 Processor -- start 2024-08-15T17:59:32.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8bf59e700 1 -- start start 2024-08-15T17:59:32.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8bf59e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff8c0156d80 0x7ff8c00aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8bf59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff8c00af170 0x7ff8c0067840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8bf59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c00cc8c0 con 0x7ff8c0067d80 2024-08-15T17:59:32.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8bf59e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c00af5c0 con 0x7ff8c0156d80 2024-08-15T17:59:32.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8bf59e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00681e0 con 0x7ff8c00af170 2024-08-15T17:59:32.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8be59c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff8c0156d80 0x7ff8c00aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8be59c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff8c0156d80 0x7ff8c00aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35572/0 (socket says 172.21.15.38:35572) 2024-08-15T17:59:32.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8be59c700 1 -- 172.21.15.38:0/1396390316 learned_addr learned my addr 172.21.15.38:0/1396390316 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:32.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8bed9d700 1 --1- 172.21.15.38:0/1396390316 >> v1:172.21.15.104:6789/0 conn(0x7ff8c0067d80 0x7ff8c00cac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:58300/0 (socket says 172.21.15.38:58300) 2024-08-15T17:59:32.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.213+0000 7ff8bdd9b700 1 --2- 172.21.15.38:0/1396390316 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff8c00af170 0x7ff8c0067840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bd59a700 1 -- 172.21.15.38:0/1396390316 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950219634 0 0) 0x7ff8c00cc8c0 con 0x7ff8c0067d80 2024-08-15T17:59:32.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bd59a700 1 -- 172.21.15.38:0/1396390316 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8a0003570 con 0x7ff8c0067d80 2024-08-15T17:59:32.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8be59c700 1 -- 172.21.15.38:0/1396390316 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff8c00af170 msgr2=0x7ff8c0067840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:32.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8be59c700 1 --2- 172.21.15.38:0/1396390316 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff8c00af170 0x7ff8c0067840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8be59c700 1 -- 172.21.15.38:0/1396390316 >> v1:172.21.15.104:6789/0 conn(0x7ff8c0067d80 legacy=0x7ff8c00cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8be59c700 1 -- 172.21.15.38:0/1396390316 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c0068450 con 0x7ff8c0156d80 2024-08-15T17:59:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8be59c700 1 --2- 172.21.15.38:0/1396390316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c0156d80 0x7ff8c00aec30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00099b0 tx=0x7ff8b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e30c7d2d05dbd9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bd59a700 1 -- 172.21.15.38:0/1396390316 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b000b680 con 0x7ff8c0156d80 2024-08-15T17:59:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bd59a700 1 -- 172.21.15.38:0/1396390316 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b000b7e0 con 0x7ff8c0156d80 2024-08-15T17:59:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bd59a700 1 -- 172.21.15.38:0/1396390316 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b000bb80 con 0x7ff8c0156d80 2024-08-15T17:59:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bf59e700 1 -- 172.21.15.38:0/1396390316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c0156d80 msgr2=0x7ff8c00aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bf59e700 1 --2- 172.21.15.38:0/1396390316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c0156d80 0x7ff8c00aec30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00099b0 tx=0x7ff8b0009250 comp rx=0 tx=0).stop 2024-08-15T17:59:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bf59e700 1 -- 172.21.15.38:0/1396390316 shutdown_connections 2024-08-15T17:59:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bf59e700 1 --2- 172.21.15.38:0/1396390316 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff8c00af170 0x7ff8c0067840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bf59e700 1 --2- 172.21.15.38:0/1396390316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c0156d80 0x7ff8c00aec30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bf59e700 1 -- 172.21.15.38:0/1396390316 >> 172.21.15.38:0/1396390316 conn(0x7ff8c006e070 msgr2=0x7ff8c00b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bf59e700 1 -- 172.21.15.38:0/1396390316 shutdown_connections 2024-08-15T17:59:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.217+0000 7ff8bf59e700 1 -- 172.21.15.38:0/1396390316 wait complete. 2024-08-15T17:59:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bf59e700 1 Processor -- start 2024-08-15T17:59:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bf59e700 1 -- start start 2024-08-15T17:59:32.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bf59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c0067d80 0x7ff8c014e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bf59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff8c00af170 0x7ff8c00ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bf59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c0156d80 0x7ff8c01508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bf59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c016ca70 con 0x7ff8c0067d80 2024-08-15T17:59:32.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bf59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c016cd50 con 0x7ff8c0156d80 2024-08-15T17:59:32.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bf59e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c016d030 con 0x7ff8c00af170 2024-08-15T17:59:32.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8be59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c0067d80 0x7ff8c014e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8be59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c0067d80 0x7ff8c014e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35586/0 (socket says 172.21.15.38:35586) 2024-08-15T17:59:32.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8be59c700 1 -- 172.21.15.38:0/2357018641 learned_addr learned my addr 172.21.15.38:0/2357018641 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bed9d700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c0156d80 0x7ff8c01508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bdd9b700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff8c00af170 0x7ff8c00ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8be59c700 1 -- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff8c00af170 msgr2=0x7ff8c00ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8be59c700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff8c00af170 0x7ff8c00ad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8be59c700 1 -- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c0156d80 msgr2=0x7ff8c01508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8be59c700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c0156d80 0x7ff8c01508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8be59c700 1 -- 172.21.15.38:0/2357018641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b00094f0 con 0x7ff8c0067d80 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bdd9b700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff8c00af170 0x7ff8c00ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8be59c700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c0067d80 0x7ff8c014e010 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009290 tx=0x7ff8b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8b67fc700 1 -- 172.21.15.38:0/2357018641 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b000b950 con 0x7ff8c0067d80 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8b67fc700 1 -- 172.21.15.38:0/2357018641 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b000bab0 con 0x7ff8c0067d80 2024-08-15T17:59:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bf59e700 1 -- 172.21.15.38:0/2357018641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c016d310 con 0x7ff8c0067d80 2024-08-15T17:59:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8b67fc700 1 -- 172.21.15.38:0/2357018641 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b001a680 con 0x7ff8c0067d80 2024-08-15T17:59:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.221+0000 7ff8bf59e700 1 -- 172.21.15.38:0/2357018641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c0154a40 con 0x7ff8c0067d80 2024-08-15T17:59:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.225+0000 7ff8bf59e700 1 -- 172.21.15.38:0/2357018641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c00638f0 con 0x7ff8c0067d80 2024-08-15T17:59:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.225+0000 7ff8b67fc700 1 -- 172.21.15.38:0/2357018641 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff8b001aa30 con 0x7ff8c0067d80 2024-08-15T17:59:32.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.225+0000 7ff8b67fc700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff8a4060860 0x7ff8a4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.225+0000 7ff8b67fc700 1 -- 172.21.15.38:0/2357018641 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff8b007e840 con 0x7ff8c0067d80 2024-08-15T17:59:32.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.225+0000 7ff8bdd9b700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff8a4060860 0x7ff8a4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.225+0000 7ff8bdd9b700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff8a4060860 0x7ff8a4062d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8006f00 tx=0x7ff8a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:32.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.237+0000 7ff8b67fc700 1 -- 172.21.15.38:0/2357018641 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff8b004ec90 con 0x7ff8c0067d80 2024-08-15T17:59:32.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.425+0000 7ff8bf59e700 1 -- 172.21.15.38:0/2357018641 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8c006ae60 con 0x7ff8a4060860 2024-08-15T17:59:32.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.429+0000 7ff8b67fc700 1 -- 172.21.15.38:0/2357018641 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff8c006ae60 con 0x7ff8a4060860 2024-08-15T17:59:32.428 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-08-15T17:59:32.429 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:32.429 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephfs" 2024-08-15T17:59:32.429 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:32.429 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-08-15T17:59:32.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 -- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff8a4060860 msgr2=0x7ff8a4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff8a4060860 0x7ff8a4062d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8006f00 tx=0x7ff8a8008040 comp rx=0 tx=0).stop 2024-08-15T17:59:32.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 -- 172.21.15.38:0/2357018641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c0067d80 msgr2=0x7ff8c014e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c0067d80 0x7ff8c014e010 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009290 tx=0x7ff8b0017000 comp rx=0 tx=0).stop 2024-08-15T17:59:32.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 -- 172.21.15.38:0/2357018641 shutdown_connections 2024-08-15T17:59:32.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff8a4060860 0x7ff8a4062d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c0156d80 0x7ff8c01508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff8c00af170 0x7ff8c00ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 --2- 172.21.15.38:0/2357018641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c0067d80 0x7ff8c014e010 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 -- 172.21.15.38:0/2357018641 >> 172.21.15.38:0/2357018641 conn(0x7ff8c006e070 msgr2=0x7ff8c00cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:32.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 -- 172.21.15.38:0/2357018641 shutdown_connections 2024-08-15T17:59:32.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.433+0000 7ff8bf59e700 1 -- 172.21.15.38:0/2357018641 wait complete. 2024-08-15T17:59:32.443 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-08-15T17:59:32.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d57385700 1 Processor -- start 2024-08-15T17:59:32.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d57385700 1 -- start start 2024-08-15T17:59:32.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d57385700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d50067400 0x7f3d501593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d57385700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d500cf670 con 0x7f3d500cfb00 2024-08-15T17:59:32.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d57385700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d500cea60 con 0x7f3d50067030 2024-08-15T17:59:32.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d57385700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d50159910 con 0x7f3d50067400 2024-08-15T17:59:32.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d55b82700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f3d500cfb00 0x7f3d50158cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:46318/0 (socket says 172.21.15.38:46318) 2024-08-15T17:59:32.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d55b82700 1 -- 172.21.15.38:0/2340606560 learned_addr learned my addr 172.21.15.38:0/2340606560 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:32.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d56b84700 1 --2- 172.21.15.38:0/2340606560 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d50067400 0x7f3d501593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d55381700 1 -- 172.21.15.38:0/2340606560 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252238845 0 0) 0x7f3d500cf670 con 0x7f3d500cfb00 2024-08-15T17:59:32.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d55381700 1 -- 172.21.15.38:0/2340606560 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d3c003570 con 0x7f3d500cfb00 2024-08-15T17:59:32.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d55381700 1 -- 172.21.15.38:0/2340606560 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657741839 0 0) 0x7f3d500cea60 con 0x7f3d50067030 2024-08-15T17:59:32.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.565+0000 7f3d55381700 1 -- 172.21.15.38:0/2340606560 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d500cf670 con 0x7f3d50067030 2024-08-15T17:59:32.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d56b84700 1 -- 172.21.15.38:0/2340606560 >> v1:172.21.15.104:6789/0 conn(0x7f3d50067030 legacy=0x7f3d500cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d56b84700 1 -- 172.21.15.38:0/2340606560 >> v1:172.21.15.38:6789/0 conn(0x7f3d500cfb00 legacy=0x7f3d50158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d56b84700 1 -- 172.21.15.38:0/2340606560 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d500ced40 con 0x7f3d50067400 2024-08-15T17:59:32.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d56b84700 1 --2- 172.21.15.38:0/2340606560 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d50067400 0x7f3d501593d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009e90 tx=0x7f3d48009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1cbabc41dc8bcd0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:32.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d55381700 1 -- 172.21.15.38:0/2340606560 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800e070 con 0x7f3d50067400 2024-08-15T17:59:32.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d55381700 1 -- 172.21.15.38:0/2340606560 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4800bb80 con 0x7f3d50067400 2024-08-15T17:59:32.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d55381700 1 -- 172.21.15.38:0/2340606560 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d48012430 con 0x7f3d50067400 2024-08-15T17:59:32.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d57385700 1 -- 172.21.15.38:0/2340606560 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d50067400 msgr2=0x7f3d501593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d57385700 1 --2- 172.21.15.38:0/2340606560 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d50067400 0x7f3d501593d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009e90 tx=0x7f3d48009780 comp rx=0 tx=0).stop 2024-08-15T17:59:32.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d57385700 1 -- 172.21.15.38:0/2340606560 shutdown_connections 2024-08-15T17:59:32.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d57385700 1 --2- 172.21.15.38:0/2340606560 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d50067400 0x7f3d501593d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d57385700 1 -- 172.21.15.38:0/2340606560 >> 172.21.15.38:0/2340606560 conn(0x7f3d500d0b50 msgr2=0x7f3d500ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:32.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d57385700 1 -- 172.21.15.38:0/2340606560 shutdown_connections 2024-08-15T17:59:32.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.569+0000 7f3d57385700 1 -- 172.21.15.38:0/2340606560 wait complete. 2024-08-15T17:59:32.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d57385700 1 Processor -- start 2024-08-15T17:59:32.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d57385700 1 -- start start 2024-08-15T17:59:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d57385700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d50067030 0x7f3d50156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d57385700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d50067400 0x7f3d500b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d57385700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d500cfb00 0x7f3d50157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d57385700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d501575a0 con 0x7f3d500cfb00 2024-08-15T17:59:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d57385700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d50156990 con 0x7f3d50067400 2024-08-15T17:59:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d57385700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d50156c70 con 0x7f3d50067030 2024-08-15T17:59:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d56b84700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d500cfb00 0x7f3d50157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d56b84700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d500cfb00 0x7f3d50157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35596/0 (socket says 172.21.15.38:35596) 2024-08-15T17:59:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d56b84700 1 -- 172.21.15.38:0/1361058826 learned_addr learned my addr 172.21.15.38:0/1361058826 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d55b82700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d50067400 0x7f3d500b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d56383700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d50067030 0x7f3d50156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d56b84700 1 -- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d50067030 msgr2=0x7f3d50156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d56b84700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d50067030 0x7f3d50156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d56b84700 1 -- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d50067400 msgr2=0x7f3d500b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d56b84700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d50067400 0x7f3d500b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d56b84700 1 -- 172.21.15.38:0/1361058826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d48009a20 con 0x7f3d500cfb00 2024-08-15T17:59:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d56b84700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d500cfb00 0x7f3d50157a30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3d5015ced0 tx=0x7f3d48019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d467fc700 1 -- 172.21.15.38:0/1361058826 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4801b420 con 0x7f3d500cfb00 2024-08-15T17:59:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d467fc700 1 -- 172.21.15.38:0/1361058826 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4801b5a0 con 0x7f3d500cfb00 2024-08-15T17:59:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d57385700 1 -- 172.21.15.38:0/1361058826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d50156f50 con 0x7f3d500cfb00 2024-08-15T17:59:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d467fc700 1 -- 172.21.15.38:0/1361058826 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4801b960 con 0x7f3d500cfb00 2024-08-15T17:59:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.573+0000 7f3d57385700 1 -- 172.21.15.38:0/1361058826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d50157f70 con 0x7f3d500cfb00 2024-08-15T17:59:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.577+0000 7f3d57385700 1 -- 172.21.15.38:0/1361058826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d50157750 con 0x7f3d500cfb00 2024-08-15T17:59:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.577+0000 7f3d467fc700 1 -- 172.21.15.38:0/1361058826 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3d4801bd50 con 0x7f3d500cfb00 2024-08-15T17:59:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.577+0000 7f3d467fc700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3d34060860 0x7f3d34062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.577+0000 7f3d467fc700 1 -- 172.21.15.38:0/1361058826 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3d4807e8e0 con 0x7f3d500cfb00 2024-08-15T17:59:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.577+0000 7f3d56383700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3d34060860 0x7f3d34062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.577+0000 7f3d56383700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3d34060860 0x7f3d34062d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c007fd0 tx=0x7f3d4c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.589+0000 7f3d467fc700 1 -- 172.21.15.38:0/1361058826 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3d50157750 con 0x7f3d500cfb00 2024-08-15T17:59:32.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.753+0000 7f3d57385700 1 -- 172.21.15.38:0/1361058826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3d500638f0 con 0x7f3d500cfb00 2024-08-15T17:59:32.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.753+0000 7f3d467fc700 1 -- 172.21.15.38:0/1361058826 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3d4804ed70 con 0x7f3d500cfb00 2024-08-15T17:59:32.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 -- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3d34060860 msgr2=0x7f3d34062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3d34060860 0x7f3d34062d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c007fd0 tx=0x7f3d4c007d50 comp rx=0 tx=0).stop 2024-08-15T17:59:32.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 -- 172.21.15.38:0/1361058826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d500cfb00 msgr2=0x7f3d50157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d500cfb00 0x7f3d50157a30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3d5015ced0 tx=0x7f3d48019000 comp rx=0 tx=0).stop 2024-08-15T17:59:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 -- 172.21.15.38:0/1361058826 shutdown_connections 2024-08-15T17:59:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f3d34060860 0x7f3d34062d10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d500cfb00 0x7f3d50157a30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d50067400 0x7f3d500b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 --2- 172.21.15.38:0/1361058826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d50067030 0x7f3d50156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 -- 172.21.15.38:0/1361058826 >> 172.21.15.38:0/1361058826 conn(0x7f3d500d0b50 msgr2=0x7f3d500c5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 -- 172.21.15.38:0/1361058826 shutdown_connections 2024-08-15T17:59:32.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.757+0000 7f3d57385700 1 -- 172.21.15.38:0/1361058826 wait complete. 2024-08-15T17:59:32.767 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000845064 2024-08-15T17:59:32.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.877+0000 7f499bfff700 1 Processor -- start 2024-08-15T17:59:32.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.877+0000 7f499bfff700 1 -- start start 2024-08-15T17:59:32.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499bfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f499c0cfb00 0x7f499c067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499bfff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c0cf670 con 0x7f499c158cc0 2024-08-15T17:59:32.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499bfff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c0cea60 con 0x7f499c067030 2024-08-15T17:59:32.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499bfff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0ced40 con 0x7f499c0cfb00 2024-08-15T17:59:32.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499b7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f499c158cc0 0x7f499c1590f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:46320/0 (socket says 172.21.15.38:46320) 2024-08-15T17:59:32.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499b7fe700 1 -- 172.21.15.38:0/1380891867 learned_addr learned my addr 172.21.15.38:0/1380891867 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499a7fc700 1 --2- 172.21.15.38:0/1380891867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f499c0cfb00 0x7f499c067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f4999ffb700 1 -- 172.21.15.38:0/1380891867 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574212580 0 0) 0x7f499c0cf670 con 0x7f499c158cc0 2024-08-15T17:59:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f4999ffb700 1 -- 172.21.15.38:0/1380891867 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4988003570 con 0x7f499c158cc0 2024-08-15T17:59:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499a7fc700 1 -- 172.21.15.38:0/1380891867 >> v1:172.21.15.104:6789/0 conn(0x7f499c067030 legacy=0x7f499c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f4999ffb700 1 -- 172.21.15.38:0/1380891867 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2413473425 0 0) 0x7f4988003570 con 0x7f499c158cc0 2024-08-15T17:59:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499a7fc700 1 -- 172.21.15.38:0/1380891867 >> v1:172.21.15.38:6789/0 conn(0x7f499c158cc0 legacy=0x7f499c1590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499a7fc700 1 -- 172.21.15.38:0/1380891867 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c0cf020 con 0x7f499c0cfb00 2024-08-15T17:59:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499a7fc700 1 --2- 172.21.15.38:0/1380891867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f499c0cfb00 0x7f499c067400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4984009910 tx=0x7f49840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d2375549157dd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f4999ffb700 1 -- 172.21.15.38:0/1380891867 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49940017f0 con 0x7f499c0cfb00 2024-08-15T17:59:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f4999ffb700 1 -- 172.21.15.38:0/1380891867 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984011420 con 0x7f499c0cfb00 2024-08-15T17:59:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f4999ffb700 1 -- 172.21.15.38:0/1380891867 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49840117a0 con 0x7f499c0cfb00 2024-08-15T17:59:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499bfff700 1 -- 172.21.15.38:0/1380891867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f499c0cfb00 msgr2=0x7f499c067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499bfff700 1 --2- 172.21.15.38:0/1380891867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f499c0cfb00 0x7f499c067400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4984009910 tx=0x7f49840091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499bfff700 1 -- 172.21.15.38:0/1380891867 shutdown_connections 2024-08-15T17:59:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499bfff700 1 --2- 172.21.15.38:0/1380891867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f499c0cfb00 0x7f499c067400 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499bfff700 1 -- 172.21.15.38:0/1380891867 >> 172.21.15.38:0/1380891867 conn(0x7f499c0d0b50 msgr2=0x7f499c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499bfff700 1 -- 172.21.15.38:0/1380891867 shutdown_connections 2024-08-15T17:59:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.881+0000 7f499bfff700 1 -- 172.21.15.38:0/1380891867 wait complete. 2024-08-15T17:59:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499bfff700 1 Processor -- start 2024-08-15T17:59:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499bfff700 1 -- start start 2024-08-15T17:59:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f499c067030 0x7f499c14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499bfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f499c0cfb00 0x7f499c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c158cc0 0x7f499c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499bfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c159c10 con 0x7f499c067030 2024-08-15T17:59:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499bfff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c156560 con 0x7f499c158cc0 2024-08-15T17:59:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499bfff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c156840 con 0x7f499c0cfb00 2024-08-15T17:59:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f499c067030 0x7f499c14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f499c067030 0x7f499c14f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35606/0 (socket says 172.21.15.38:35606) 2024-08-15T17:59:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499affd700 1 -- 172.21.15.38:0/1318029623 learned_addr learned my addr 172.21.15.38:0/1318029623 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499a7fc700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f499c0cfb00 0x7f499c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.885+0000 7f499b7fe700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c158cc0 0x7f499c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f499affd700 1 -- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f499c0cfb00 msgr2=0x7f499c155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f499affd700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f499c0cfb00 0x7f499c155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f499affd700 1 -- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c158cc0 msgr2=0x7f499c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:32.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f499affd700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c158cc0 0x7f499c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:32.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f499affd700 1 -- 172.21.15.38:0/1318029623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4984009420 con 0x7f499c067030 2024-08-15T17:59:32.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f499affd700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f499c067030 0x7f499c14f210 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f498c00c810 tx=0x7f498c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:32.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f4992ffd700 1 -- 172.21.15.38:0/1318029623 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c011590 con 0x7f499c067030 2024-08-15T17:59:32.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f4992ffd700 1 -- 172.21.15.38:0/1318029623 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c011730 con 0x7f499c067030 2024-08-15T17:59:32.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f499bfff700 1 -- 172.21.15.38:0/1318029623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c156b20 con 0x7f499c067030 2024-08-15T17:59:32.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f4992ffd700 1 -- 172.21.15.38:0/1318029623 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c011af0 con 0x7f499c067030 2024-08-15T17:59:32.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f499bfff700 1 -- 172.21.15.38:0/1318029623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c14f750 con 0x7f499c067030 2024-08-15T17:59:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f499bfff700 1 -- 172.21.15.38:0/1318029623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c159a00 con 0x7f499c067030 2024-08-15T17:59:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.889+0000 7f4992ffd700 1 -- 172.21.15.38:0/1318029623 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f498c012420 con 0x7f499c067030 2024-08-15T17:59:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.893+0000 7f4992ffd700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f497c060860 0x7f497c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.893+0000 7f4992ffd700 1 -- 172.21.15.38:0/1318029623 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f498c07c0c0 con 0x7f499c067030 2024-08-15T17:59:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.893+0000 7f499a7fc700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f497c060860 0x7f497c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:32.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.893+0000 7f499a7fc700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f497c060860 0x7f497c062d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4984009860 tx=0x7f4984017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:32.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:32.897+0000 7f4992ffd700 1 -- 172.21.15.38:0/1318029623 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f499c159a00 con 0x7f499c067030 2024-08-15T17:59:33.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:33.089+0000 7f499bfff700 1 -- 172.21.15.38:0/1318029623 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000845064", "target": ["mon-mgr", ""]}) v1 -- 0x7f499c159d50 con 0x7f497c060860 2024-08-15T17:59:35.280 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:59:35.283 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:59:36.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.125+0000 7f4992ffd700 1 -- 172.21.15.38:0/1318029623 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f4984000ce0 con 0x7f497c060860 2024-08-15T17:59:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.129+0000 7f499bfff700 1 -- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f497c060860 msgr2=0x7f497c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.129+0000 7f499bfff700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f497c060860 0x7f497c062d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4984009860 tx=0x7f4984017000 comp rx=0 tx=0).stop 2024-08-15T17:59:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.129+0000 7f499bfff700 1 -- 172.21.15.38:0/1318029623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f499c067030 msgr2=0x7f499c14f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.129+0000 7f499bfff700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f499c067030 0x7f499c14f210 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f498c00c810 tx=0x7f498c007f30 comp rx=0 tx=0).stop 2024-08-15T17:59:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.129+0000 7f499bfff700 1 -- 172.21.15.38:0/1318029623 shutdown_connections 2024-08-15T17:59:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.129+0000 7f499bfff700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f497c060860 0x7f497c062d10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.129+0000 7f499bfff700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c158cc0 0x7f499c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.129+0000 7f499bfff700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f499c0cfb00 0x7f499c155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.129+0000 7f499bfff700 1 --2- 172.21.15.38:0/1318029623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f499c067030 0x7f499c14f210 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.129+0000 7f499bfff700 1 -- 172.21.15.38:0/1318029623 >> 172.21.15.38:0/1318029623 conn(0x7f499c0d0b50 msgr2=0x7f499c0cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.133+0000 7f499bfff700 1 -- 172.21.15.38:0/1318029623 shutdown_connections 2024-08-15T17:59:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.133+0000 7f499bfff700 1 -- 172.21.15.38:0/1318029623 wait complete. 2024-08-15T17:59:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:Volume created successfully (no MDS daemons created) 2024-08-15T17:59:36.143 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-08-15T17:59:36.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.249+0000 7efe3759e700 1 Processor -- start 2024-08-15T17:59:36.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.249+0000 7efe3759e700 1 -- start start 2024-08-15T17:59:36.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.249+0000 7efe3759e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efe380b14a0 0x7efe380cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.249+0000 7efe3759e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe380cbaa0 con 0x7efe38154900 2024-08-15T17:59:36.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.249+0000 7efe3759e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe38157630 con 0x7efe380cb390 2024-08-15T17:59:36.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.249+0000 7efe3759e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe38068790 con 0x7efe380b14a0 2024-08-15T17:59:36.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.249+0000 7efe35d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efe380b14a0 0x7efe380cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.249+0000 7efe35d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efe380b14a0 0x7efe380cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35610/0 (socket says 172.21.15.38:35610) 2024-08-15T17:59:36.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.249+0000 7efe35d9b700 1 -- 172.21.15.38:0/3036380772 learned_addr learned my addr 172.21.15.38:0/3036380772 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:36.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.249+0000 7efe3659c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7efe38154900 0x7efe380b0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:46334/0 (socket says 172.21.15.38:46334) 2024-08-15T17:59:36.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe35d9b700 1 -- 172.21.15.38:0/3036380772 >> v1:172.21.15.104:6789/0 conn(0x7efe380cb390 legacy=0x7efe38160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe35d9b700 1 -- 172.21.15.38:0/3036380772 >> v1:172.21.15.38:6789/0 conn(0x7efe38154900 legacy=0x7efe380b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe35d9b700 1 -- 172.21.15.38:0/3036380772 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe38157910 con 0x7efe380b14a0 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3559a700 1 -- 172.21.15.38:0/3036380772 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580345216 0 0) 0x7efe380cbaa0 con 0x7efe38154900 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe35d9b700 1 --2- 172.21.15.38:0/3036380772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe380b14a0 0x7efe380cae50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efe200099b0 tx=0x7efe20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd1257a10a4aea3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3559a700 1 -- 172.21.15.38:0/3036380772 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2000b860 con 0x7efe380b14a0 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3559a700 1 -- 172.21.15.38:0/3036380772 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2000b9c0 con 0x7efe380b14a0 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3559a700 1 -- 172.21.15.38:0/3036380772 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2000bd60 con 0x7efe380b14a0 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 -- 172.21.15.38:0/3036380772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe380b14a0 msgr2=0x7efe380cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 --2- 172.21.15.38:0/3036380772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe380b14a0 0x7efe380cae50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efe200099b0 tx=0x7efe20009250 comp rx=0 tx=0).stop 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 -- 172.21.15.38:0/3036380772 shutdown_connections 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 --2- 172.21.15.38:0/3036380772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe380b14a0 0x7efe380cae50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 -- 172.21.15.38:0/3036380772 >> 172.21.15.38:0/3036380772 conn(0x7efe380678d0 msgr2=0x7efe38067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:36.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 -- 172.21.15.38:0/3036380772 shutdown_connections 2024-08-15T17:59:36.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 -- 172.21.15.38:0/3036380772 wait complete. 2024-08-15T17:59:36.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 Processor -- start 2024-08-15T17:59:36.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 -- start start 2024-08-15T17:59:36.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe380b14a0 0x7efe38141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe380cb390 0x7efe38166f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe38154900 0x7efe380ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe38168590 con 0x7efe380b14a0 2024-08-15T17:59:36.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe381679a0 con 0x7efe380cb390 2024-08-15T17:59:36.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3759e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe38167c80 con 0x7efe38154900 2024-08-15T17:59:36.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe380b14a0 0x7efe38141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe380b14a0 0x7efe38141b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35618/0 (socket says 172.21.15.38:35618) 2024-08-15T17:59:36.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.253+0000 7efe3659c700 1 -- 172.21.15.38:0/2099832071 learned_addr learned my addr 172.21.15.38:0/2099832071 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:36.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe36d9d700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe38154900 0x7efe380ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe35d9b700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe380cb390 0x7efe38166f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe3659c700 1 -- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe38154900 msgr2=0x7efe380ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe3659c700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe38154900 0x7efe380ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe3659c700 1 -- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe380cb390 msgr2=0x7efe38166f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe3659c700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe380cb390 0x7efe38166f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe3659c700 1 -- 172.21.15.38:0/2099832071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe200094f0 con 0x7efe380b14a0 2024-08-15T17:59:36.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe35d9b700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe380cb390 0x7efe38166f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:59:36.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe3659c700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe380b14a0 0x7efe38141b40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efe2800c810 tx=0x7efe28007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:36.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe2e7fc700 1 -- 172.21.15.38:0/2099832071 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe280073d0 con 0x7efe380b14a0 2024-08-15T17:59:36.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe2e7fc700 1 -- 172.21.15.38:0/2099832071 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe28007550 con 0x7efe380b14a0 2024-08-15T17:59:36.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe3759e700 1 -- 172.21.15.38:0/2099832071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe38168220 con 0x7efe380b14a0 2024-08-15T17:59:36.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe2e7fc700 1 -- 172.21.15.38:0/2099832071 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28004a80 con 0x7efe380b14a0 2024-08-15T17:59:36.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe3759e700 1 -- 172.21.15.38:0/2099832071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe380aad30 con 0x7efe380b14a0 2024-08-15T17:59:36.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe3759e700 1 -- 172.21.15.38:0/2099832071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe38168080 con 0x7efe380b14a0 2024-08-15T17:59:36.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.257+0000 7efe2e7fc700 1 -- 172.21.15.38:0/2099832071 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7efe2801c3b0 con 0x7efe380b14a0 2024-08-15T17:59:36.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.261+0000 7efe2e7fc700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efe1c060860 0x7efe1c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.261+0000 7efe2e7fc700 1 -- 172.21.15.38:0/2099832071 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7efe2807c3e0 con 0x7efe380b14a0 2024-08-15T17:59:36.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.261+0000 7efe35d9b700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efe1c060860 0x7efe1c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.261+0000 7efe35d9b700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efe1c060860 0x7efe1c062d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efe200099b0 tx=0x7efe20017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:36.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.269+0000 7efe2e7fc700 1 -- 172.21.15.38:0/2099832071 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7efe38168080 con 0x7efe380b14a0 2024-08-15T17:59:36.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.453+0000 7efe3759e700 1 -- 172.21.15.38:0/2099832071 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7efe381686d0 con 0x7efe1c060860 2024-08-15T17:59:36.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe2e7fc700 1 -- 172.21.15.38:0/2099832071 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7efe2000bb30 con 0x7efe1c060860 2024-08-15T17:59:36.455 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-08-15T17:59:36.455 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:36.455 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephfs" 2024-08-15T17:59:36.455 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-08-15T17:59:36.455 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-08-15T17:59:36.455 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volume_0000000000845064" 2024-08-15T17:59:36.456 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-08-15T17:59:36.456 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-08-15T17:59:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 -- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efe1c060860 msgr2=0x7efe1c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efe1c060860 0x7efe1c062d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efe200099b0 tx=0x7efe20017000 comp rx=0 tx=0).stop 2024-08-15T17:59:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 -- 172.21.15.38:0/2099832071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe380b14a0 msgr2=0x7efe38141b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe380b14a0 0x7efe38141b40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efe2800c810 tx=0x7efe28007d90 comp rx=0 tx=0).stop 2024-08-15T17:59:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 -- 172.21.15.38:0/2099832071 shutdown_connections 2024-08-15T17:59:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7efe1c060860 0x7efe1c062d10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe38154900 0x7efe380ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe380cb390 0x7efe38166f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 --2- 172.21.15.38:0/2099832071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe380b14a0 0x7efe38141b40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 -- 172.21.15.38:0/2099832071 >> 172.21.15.38:0/2099832071 conn(0x7efe380678d0 msgr2=0x7efe381522e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:36.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 -- 172.21.15.38:0/2099832071 shutdown_connections 2024-08-15T17:59:36.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.457+0000 7efe3759e700 1 -- 172.21.15.38:0/2099832071 wait complete. 2024-08-15T17:59:36.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2024-08-15T17:59:36.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.585+0000 7f5570b99700 1 Processor -- start 2024-08-15T17:59:36.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.585+0000 7f5570b99700 1 -- start start 2024-08-15T17:59:36.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.585+0000 7f5570b99700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 0x7f556c0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.585+0000 7f5570b99700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f556c0cd080 0x7f556c0ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.585+0000 7f5570b99700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f556c0cf1c0 con 0x7f556c0ced10 2024-08-15T17:59:36.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.585+0000 7f5570b99700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556c0cf460 con 0x7f556c0cd080 2024-08-15T17:59:36.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.585+0000 7f5570b99700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0cf700 con 0x7f556c1591e0 2024-08-15T17:59:36.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.585+0000 7f556ad9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f556c0cd080 0x7f556c0ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.585+0000 7f556ad9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f556c0cd080 0x7f556c0ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54350/0 (socket says 172.21.15.38:54350) 2024-08-15T17:59:36.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.585+0000 7f556ad9d700 1 -- 172.21.15.38:0/3220194560 learned_addr learned my addr 172.21.15.38:0/3220194560 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:36.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.585+0000 7f556a59c700 1 --2- 172.21.15.38:0/3220194560 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 0x7f556c0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f556ad9d700 1 -- 172.21.15.38:0/3220194560 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 msgr2=0x7f556c0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:36.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f556ad9d700 1 --2- 172.21.15.38:0/3220194560 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 0x7f556c0ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f556ad9d700 1 -- 172.21.15.38:0/3220194560 >> v1:172.21.15.104:6789/0 conn(0x7f556c0ced10 legacy=0x7f556c0cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f556ad9d700 1 -- 172.21.15.38:0/3220194560 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c0cf9c0 con 0x7f556c0cd080 2024-08-15T17:59:36.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f556a59c700 1 --2- 172.21.15.38:0/3220194560 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 0x7f556c0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:36.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f556ad9d700 1 --2- 172.21.15.38:0/3220194560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f556c0cd080 0x7f556c0ad2d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5564009f20 tx=0x7f5564009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692e1c74ffe5ccf4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f556959a700 1 -- 172.21.15.38:0/3220194560 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556400e070 con 0x7f556c0cd080 2024-08-15T17:59:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f556959a700 1 -- 172.21.15.38:0/3220194560 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556400ba80 con 0x7f556c0cd080 2024-08-15T17:59:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f556959a700 1 -- 172.21.15.38:0/3220194560 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55640092e0 con 0x7f556c0cd080 2024-08-15T17:59:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f5570b99700 1 -- 172.21.15.38:0/3220194560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f556c0cd080 msgr2=0x7f556c0ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f5570b99700 1 --2- 172.21.15.38:0/3220194560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f556c0cd080 0x7f556c0ad2d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5564009f20 tx=0x7f5564009840 comp rx=0 tx=0).stop 2024-08-15T17:59:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f5570b99700 1 -- 172.21.15.38:0/3220194560 shutdown_connections 2024-08-15T17:59:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f5570b99700 1 --2- 172.21.15.38:0/3220194560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f556c0cd080 0x7f556c0ad2d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f5570b99700 1 --2- 172.21.15.38:0/3220194560 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 0x7f556c0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f5570b99700 1 -- 172.21.15.38:0/3220194560 >> 172.21.15.38:0/3220194560 conn(0x7f556c089bc0 msgr2=0x7f556c0994e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f5570b99700 1 -- 172.21.15.38:0/3220194560 shutdown_connections 2024-08-15T17:59:36.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.589+0000 7f5570b99700 1 -- 172.21.15.38:0/3220194560 wait complete. 2024-08-15T17:59:36.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f5570b99700 1 Processor -- start 2024-08-15T17:59:36.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f5570b99700 1 -- start start 2024-08-15T17:59:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f5570b99700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f556c0cd080 0x7f556c15bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f5570b99700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f556c0ced10 0x7f556c0c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f5570b99700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 0x7f556c15f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f5570b99700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0c5fa0 con 0x7f556c0ced10 2024-08-15T17:59:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f5570b99700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0c5390 con 0x7f556c0cd080 2024-08-15T17:59:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f5570b99700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0c5670 con 0x7f556c1591e0 2024-08-15T17:59:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f5569d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f556c0ced10 0x7f556c0c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f5569d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f556c0ced10 0x7f556c0c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54366/0 (socket says 172.21.15.38:54366) 2024-08-15T17:59:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f5569d9b700 1 -- 172.21.15.38:0/2281903442 learned_addr learned my addr 172.21.15.38:0/2281903442 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f556a59c700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f556c0cd080 0x7f556c15bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.593+0000 7f556ad9d700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 0x7f556c15f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f5569d9b700 1 -- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 msgr2=0x7f556c15f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f5569d9b700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 0x7f556c15f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f5569d9b700 1 -- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f556c0cd080 msgr2=0x7f556c15bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f5569d9b700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f556c0cd080 0x7f556c15bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f5569d9b700 1 -- 172.21.15.38:0/2281903442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5564009ae0 con 0x7f556c0ced10 2024-08-15T17:59:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f556ad9d700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 0x7f556c15f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:59:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f5569d9b700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f556c0ced10 0x7f556c0c6430 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f555c00c590 tx=0x7f555c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f55627fc700 1 -- 172.21.15.38:0/2281903442 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c011590 con 0x7f556c0ced10 2024-08-15T17:59:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f55627fc700 1 -- 172.21.15.38:0/2281903442 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555c011710 con 0x7f556c0ced10 2024-08-15T17:59:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f55627fc700 1 -- 172.21.15.38:0/2281903442 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c011ad0 con 0x7f556c0ced10 2024-08-15T17:59:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f5570b99700 1 -- 172.21.15.38:0/2281903442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c0c5950 con 0x7f556c0ced10 2024-08-15T17:59:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f5570b99700 1 -- 172.21.15.38:0/2281903442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c15c410 con 0x7f556c0ced10 2024-08-15T17:59:36.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f55627fc700 1 -- 172.21.15.38:0/2281903442 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f555c01b3b0 con 0x7f556c0ced10 2024-08-15T17:59:36.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f5570b99700 1 -- 172.21.15.38:0/2281903442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c0645c0 con 0x7f556c0ced10 2024-08-15T17:59:36.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f55627fc700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f554c060860 0x7f554c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f55627fc700 1 -- 172.21.15.38:0/2281903442 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f555c07c450 con 0x7f556c0ced10 2024-08-15T17:59:36.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.597+0000 7f556a59c700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f554c060860 0x7f554c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.601+0000 7f556a59c700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f554c060860 0x7f554c062d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5554006f00 tx=0x7f5554008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:36.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.605+0000 7f55627fc700 1 -- 172.21.15.38:0/2281903442 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f555c04c520 con 0x7f556c0ced10 2024-08-15T17:59:36.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 -- 172.21.15.38:0/2281903442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f556c0c60e0 con 0x7f556c0ced10 2024-08-15T17:59:36.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f55627fc700 1 -- 172.21.15.38:0/2281903442 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v35) v1 ==== 97+0+8050 (secure 0 0 0) 0x7f555c04fb40 con 0x7f556c0ced10 2024-08-15T17:59:36.768 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:36.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 -- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f554c060860 msgr2=0x7f554c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f554c060860 0x7f554c062d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5554006f00 tx=0x7f5554008040 comp rx=0 tx=0).stop 2024-08-15T17:59:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 -- 172.21.15.38:0/2281903442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f556c0ced10 msgr2=0x7f556c0c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f556c0ced10 0x7f556c0c6430 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f555c00c590 tx=0x7f555c00c8a0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 -- 172.21.15.38:0/2281903442 shutdown_connections 2024-08-15T17:59:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f554c060860 0x7f554c062d10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f556c1591e0 0x7f556c15f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f556c0ced10 0x7f556c0c6430 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 --2- 172.21.15.38:0/2281903442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f556c0cd080 0x7f556c15bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 -- 172.21.15.38:0/2281903442 >> 172.21.15.38:0/2281903442 conn(0x7f556c089bc0 msgr2=0x7f556c068bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.769+0000 7f5570b99700 1 -- 172.21.15.38:0/2281903442 shutdown_connections 2024-08-15T17:59:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.773+0000 7f5570b99700 1 -- 172.21.15.38:0/2281903442 wait complete. 2024-08-15T17:59:36.782 INFO:teuthology.orchestra.run.smithi038.stdout:[{"pool_id":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool_id":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:59:18.793510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool_id":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:59:19.328139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool_id":6,"pool_name":"cephfs.volume_0000000000845064.meta","create_time":"2024-08-15T17:59:33.132275+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000845064"}}},{"pool_id":7,"pool_name":"cephfs.volume_0000000000845064.data","create_time":"2024-08-15T17:59:34.143101+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000845064"}}}] 2024-08-15T17:59:36.783 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get volume_0000000000845064 --format=json 2024-08-15T17:59:36.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.889+0000 7ff50abbc700 1 Processor -- start 2024-08-15T17:59:36.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.889+0000 7ff50abbc700 1 -- start start 2024-08-15T17:59:36.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff50abbc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff504154900 0x7ff5040b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff50abbc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff5040cb560 0x7ff504160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff50abbc700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5040b1730 con 0x7ff5040b12d0 2024-08-15T17:59:36.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff50abbc700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff504068790 con 0x7ff5040cb560 2024-08-15T17:59:36.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff50abbc700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5041583f0 con 0x7ff504154900 2024-08-15T17:59:36.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff50a3bb700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff5040cb560 0x7ff504160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff5093b9700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff5040b12d0 0x7ff5040cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45486/0 (socket says 172.21.15.38:45486) 2024-08-15T17:59:36.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff5093b9700 1 -- 172.21.15.38:0/2667811219 learned_addr learned my addr 172.21.15.38:0/2667811219 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:36.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff509bba700 1 --2- 172.21.15.38:0/2667811219 >> v2:172.21.15.104:3300/0 conn(0x7ff504154900 0x7ff5040b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff50a3bb700 1 -- 172.21.15.38:0/2667811219 >> v2:172.21.15.104:3300/0 conn(0x7ff504154900 msgr2=0x7ff5040b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:36.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff50a3bb700 1 --2- 172.21.15.38:0/2667811219 >> v2:172.21.15.104:3300/0 conn(0x7ff504154900 0x7ff5040b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff50a3bb700 1 -- 172.21.15.38:0/2667811219 >> v1:172.21.15.38:6789/0 conn(0x7ff5040b12d0 legacy=0x7ff5040cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff50a3bb700 1 -- 172.21.15.38:0/2667811219 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5040cbaa0 con 0x7ff5040cb560 2024-08-15T17:59:36.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.893+0000 7ff50a3bb700 1 --2- 172.21.15.38:0/2667811219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5040cb560 0x7ff504160ac0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff504064e60 tx=0x7ff4fc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991aa9c4e69708f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:36.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.897+0000 7ff508bb8700 1 -- 172.21.15.38:0/2667811219 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00e070 con 0x7ff5040cb560 2024-08-15T17:59:36.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.897+0000 7ff508bb8700 1 -- 172.21.15.38:0/2667811219 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc00bb60 con 0x7ff5040cb560 2024-08-15T17:59:36.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.897+0000 7ff508bb8700 1 -- 172.21.15.38:0/2667811219 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc012430 con 0x7ff5040cb560 2024-08-15T17:59:36.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.897+0000 7ff50abbc700 1 -- 172.21.15.38:0/2667811219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5040cb560 msgr2=0x7ff504160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.897+0000 7ff50abbc700 1 --2- 172.21.15.38:0/2667811219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5040cb560 0x7ff504160ac0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff504064e60 tx=0x7ff4fc009780 comp rx=0 tx=0).stop 2024-08-15T17:59:36.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.897+0000 7ff50abbc700 1 -- 172.21.15.38:0/2667811219 shutdown_connections 2024-08-15T17:59:36.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.897+0000 7ff50abbc700 1 --2- 172.21.15.38:0/2667811219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5040cb560 0x7ff504160ac0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.897+0000 7ff50abbc700 1 --2- 172.21.15.38:0/2667811219 >> v2:172.21.15.104:3300/0 conn(0x7ff504154900 0x7ff5040b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.897+0000 7ff50abbc700 1 -- 172.21.15.38:0/2667811219 >> 172.21.15.38:0/2667811219 conn(0x7ff5040678d0 msgr2=0x7ff504067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:36.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.897+0000 7ff50abbc700 1 -- 172.21.15.38:0/2667811219 shutdown_connections 2024-08-15T17:59:36.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.897+0000 7ff50abbc700 1 -- 172.21.15.38:0/2667811219 wait complete. 2024-08-15T17:59:36.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50abbc700 1 Processor -- start 2024-08-15T17:59:36.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50abbc700 1 -- start start 2024-08-15T17:59:36.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50abbc700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff5040b12d0 0x7ff5041597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50abbc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5040cb560 0x7ff5040aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50abbc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff504154900 0x7ff504145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50abbc700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040b0250 con 0x7ff504154900 2024-08-15T17:59:36.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50abbc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040af640 con 0x7ff5040cb560 2024-08-15T17:59:36.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50abbc700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040af920 con 0x7ff5040b12d0 2024-08-15T17:59:36.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50a3bb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff504154900 0x7ff504145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50a3bb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff504154900 0x7ff504145130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54392/0 (socket says 172.21.15.38:54392) 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50a3bb700 1 -- 172.21.15.38:0/2479028106 learned_addr learned my addr 172.21.15.38:0/2479028106 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff509bba700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff5040b12d0 0x7ff5041597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff5093b9700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5040cb560 0x7ff5040aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50a3bb700 1 -- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff5040b12d0 msgr2=0x7ff5041597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50a3bb700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff5040b12d0 0x7ff5041597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50a3bb700 1 -- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5040cb560 msgr2=0x7ff5040aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50a3bb700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5040cb560 0x7ff5040aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50a3bb700 1 -- 172.21.15.38:0/2479028106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc009a20 con 0x7ff504154900 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50a3bb700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff504154900 0x7ff504145130 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009fd0 tx=0x7ff4fc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff4f9ffb700 1 -- 172.21.15.38:0/2479028106 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc027470 con 0x7ff504154900 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff4f9ffb700 1 -- 172.21.15.38:0/2479028106 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc0275f0 con 0x7ff504154900 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50abbc700 1 -- 172.21.15.38:0/2479028106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5040afc00 con 0x7ff504154900 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff4f9ffb700 1 -- 172.21.15.38:0/2479028106 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0279b0 con 0x7ff504154900 2024-08-15T17:59:36.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.901+0000 7ff50abbc700 1 -- 172.21.15.38:0/2479028106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5040ab260 con 0x7ff504154900 2024-08-15T17:59:36.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.905+0000 7ff50abbc700 1 -- 172.21.15.38:0/2479028106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5040b0400 con 0x7ff504154900 2024-08-15T17:59:36.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.905+0000 7ff4f9ffb700 1 -- 172.21.15.38:0/2479028106 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff4fc024070 con 0x7ff504154900 2024-08-15T17:59:36.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.905+0000 7ff4f9ffb700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff4e8060860 0x7ff4e8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:36.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.905+0000 7ff4f9ffb700 1 -- 172.21.15.38:0/2479028106 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7ff4fc07f910 con 0x7ff504154900 2024-08-15T17:59:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.905+0000 7ff509bba700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff4e8060860 0x7ff4e8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.905+0000 7ff509bba700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff4e8060860 0x7ff4e8062d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff500009d10 tx=0x7ff5000091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:36.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:36.909+0000 7ff4f9ffb700 1 -- 172.21.15.38:0/2479028106 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff5040b0400 con 0x7ff504154900 2024-08-15T17:59:37.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.117+0000 7ff50abbc700 1 -- 172.21.15.38:0/2479028106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "volume_0000000000845064", "format": "json"} v 0) v1 -- 0x7ff5040643a0 con 0x7ff504154900 2024-08-15T17:59:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.121+0000 7ff4f9ffb700 1 -- 172.21.15.38:0/2479028106 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "volume_0000000000845064", "format": "json"}]=0 v18) v1 ==== 110+0+1844 (secure 0 0 0) 0x7ff4fc02c750 con 0x7ff504154900 2024-08-15T17:59:37.123 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T17:59:37.124 INFO:teuthology.orchestra.run.smithi038.stdout:{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:35.121185+0000","modified":"2024-08-15T17:59:36.892161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4981":{"gid":4981,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6835/875828080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":875828080},{"type":"v1","addr":"172.21.15.104:6835","nonce":875828080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000845064","balancer":"","standby_count_wanted":1},"id":3} 2024-08-15T17:59:37.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 -- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff4e8060860 msgr2=0x7ff4e8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:37.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff4e8060860 0x7ff4e8062d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff500009d10 tx=0x7ff5000091a0 comp rx=0 tx=0).stop 2024-08-15T17:59:37.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 -- 172.21.15.38:0/2479028106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff504154900 msgr2=0x7ff504145130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:37.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff504154900 0x7ff504145130 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009fd0 tx=0x7ff4fc019000 comp rx=0 tx=0).stop 2024-08-15T17:59:37.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 -- 172.21.15.38:0/2479028106 shutdown_connections 2024-08-15T17:59:37.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff4e8060860 0x7ff4e8062d10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:37.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff504154900 0x7ff504145130 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:37.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5040cb560 0x7ff5040aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:37.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 --2- 172.21.15.38:0/2479028106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff5040b12d0 0x7ff5041597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:37.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 -- 172.21.15.38:0/2479028106 >> 172.21.15.38:0/2479028106 conn(0x7ff5040678d0 msgr2=0x7ff504068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:37.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 -- 172.21.15.38:0/2479028106 shutdown_connections 2024-08-15T17:59:37.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.133+0000 7ff50abbc700 1 -- 172.21.15.38:0/2479028106 wait complete. 2024-08-15T17:59:37.146 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000845064 --yes-i-really-mean-it 2024-08-15T17:59:37.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.253+0000 7f1699372700 1 Processor -- start 2024-08-15T17:59:37.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.253+0000 7f1699372700 1 -- start start 2024-08-15T17:59:37.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1699372700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1694068b20 0x7f1694160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:37.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1699372700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16940ae490 con 0x7f16941552a0 2024-08-15T17:59:37.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1699372700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1694156d90 con 0x7f16940686a0 2024-08-15T17:59:37.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1699372700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16940ae920 con 0x7f1694068b20 2024-08-15T17:59:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1693fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f16941552a0 0x7f16940b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45502/0 (socket says 172.21.15.38:45502) 2024-08-15T17:59:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1693fff700 1 -- 172.21.15.38:0/2014785829 learned_addr learned my addr 172.21.15.38:0/2014785829 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T17:59:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1698b71700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1694068b20 0x7f1694160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957540211 0 0) 0x7f16940ae490 con 0x7f16941552a0 2024-08-15T17:59:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1680003570 con 0x7f16941552a0 2024-08-15T17:59:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992752916 0 0) 0x7f1694156d90 con 0x7f16940686a0 2024-08-15T17:59:37.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16940ae490 con 0x7f16940686a0 2024-08-15T17:59:37.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4129322516 0 0) 0x7f1680003570 con 0x7f16941552a0 2024-08-15T17:59:37.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1694068b20 msgr2=0x7f1694160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:37.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1694068b20 0x7f1694160530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:37.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 >> v1:172.21.15.104:6789/0 conn(0x7f16940686a0 legacy=0x7f1694155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:37.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16940ad880 con 0x7f16941552a0 2024-08-15T17:59:37.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f1684001470 con 0x7f16941552a0 2024-08-15T17:59:37.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 >> v1:172.21.15.38:6789/0 conn(0x7f16941552a0 legacy=0x7f16940b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:37.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1680003c20 0x7f1680004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:37.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1680004810 0x7f1680004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1694068b20 0x7f16800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168000bb40 con 0x7f1680003c20 2024-08-15T17:59:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168000bcb0 con 0x7f1680004810 2024-08-15T17:59:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168000be70 con 0x7f1694068b20 2024-08-15T17:59:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1693fff700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1680003c20 0x7f1680004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f168bfff700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1680004810 0x7f1680004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1698b71700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1694068b20 0x7f16800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1693fff700 1 -- 172.21.15.38:0/2014785829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1694068b20 msgr2=0x7f16800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1693fff700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1694068b20 0x7f16800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1693fff700 1 -- 172.21.15.38:0/2014785829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1680004810 msgr2=0x7f1680004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1693fff700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1680004810 0x7f1680004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1693fff700 1 -- 172.21.15.38:0/2014785829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1680003570 con 0x7f1680003c20 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.257+0000 7f1693fff700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1680003c20 0x7f1680004010 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f168400c2d0 tx=0x7f1684001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f3f452dc8b7c72 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168400cba0 con 0x7f1680003c20 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168400cd00 con 0x7f1680003c20 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f16937fe700 1 -- 172.21.15.38:0/2014785829 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1684005b70 con 0x7f1680003c20 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f1699372700 1 -- 172.21.15.38:0/2014785829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1680003c20 msgr2=0x7f1680004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f1699372700 1 --2- 172.21.15.38:0/2014785829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1680003c20 0x7f1680004010 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f168400c2d0 tx=0x7f1684001e10 comp rx=0 tx=0).stop 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f1698b71700 1 -- 172.21.15.38:0/2014785829 reap_dead start 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f1699372700 1 -- 172.21.15.38:0/2014785829 shutdown_connections 2024-08-15T17:59:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f1699372700 1 -- 172.21.15.38:0/2014785829 >> 172.21.15.38:0/2014785829 conn(0x7f169406e350 msgr2=0x7f169406b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:59:37.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f1699372700 1 -- 172.21.15.38:0/2014785829 shutdown_connections 2024-08-15T17:59:37.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f1699372700 1 -- 172.21.15.38:0/2014785829 wait complete. 2024-08-15T17:59:37.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f1699372700 1 Processor -- start 2024-08-15T17:59:37.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.261+0000 7f1699372700 1 -- start start 2024-08-15T17:59:37.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1699372700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1680003c20 0x7f1694142b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:37.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1699372700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16940a9c50 0x7f16940aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:37.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1699372700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16940aa600 0x7f16940aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:37.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1699372700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169414bb90 con 0x7f16940aa600 2024-08-15T17:59:37.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1699372700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169414afd0 con 0x7f16940a9c50 2024-08-15T17:59:37.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1699372700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16940abd80 con 0x7f1680003c20 2024-08-15T17:59:37.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1698b71700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16940aa600 0x7f16940aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:37.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f168bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16940a9c50 0x7f16940aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:37.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1698b71700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16940aa600 0x7f16940aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54412/0 (socket says 172.21.15.38:54412) 2024-08-15T17:59:37.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1698b71700 1 -- 172.21.15.38:0/448214528 learned_addr learned my addr 172.21.15.38:0/448214528 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T17:59:37.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1693fff700 1 --2- 172.21.15.38:0/448214528 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1680003c20 0x7f1694142b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1698b71700 1 -- 172.21.15.38:0/448214528 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1680003c20 msgr2=0x7f1694142b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1698b71700 1 --2- 172.21.15.38:0/448214528 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1680003c20 0x7f1694142b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1698b71700 1 -- 172.21.15.38:0/448214528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16940a9c50 msgr2=0x7f16940aa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1698b71700 1 --2- 172.21.15.38:0/448214528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16940a9c50 0x7f16940aa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1698b71700 1 -- 172.21.15.38:0/448214528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169414b260 con 0x7f16940aa600 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1698b71700 1 --2- 172.21.15.38:0/448214528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16940aa600 0x7f16940aaa70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1694069240 tx=0x7f168c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1690ff9700 1 -- 172.21.15.38:0/448214528 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c00e070 con 0x7f16940aa600 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1690ff9700 1 -- 172.21.15.38:0/448214528 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c011c30 con 0x7f16940aa600 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1690ff9700 1 -- 172.21.15.38:0/448214528 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c016470 con 0x7f16940aa600 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1699372700 1 -- 172.21.15.38:0/448214528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169414b540 con 0x7f16940aa600 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1699372700 1 -- 172.21.15.38:0/448214528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1694143050 con 0x7f16940aa600 2024-08-15T17:59:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.265+0000 7f1699372700 1 -- 172.21.15.38:0/448214528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169414c000 con 0x7f16940aa600 2024-08-15T17:59:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.269+0000 7f1690ff9700 1 -- 172.21.15.38:0/448214528 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f168c016860 con 0x7f16940aa600 2024-08-15T17:59:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.269+0000 7f1690ff9700 1 --2- 172.21.15.38:0/448214528 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1674060890 0x7f1674062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:59:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.269+0000 7f1690ff9700 1 -- 172.21.15.38:0/448214528 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f168c07d950 con 0x7f16940aa600 2024-08-15T17:59:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.269+0000 7f1693fff700 1 --2- 172.21.15.38:0/448214528 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1674060890 0x7f1674062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:59:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.269+0000 7f1693fff700 1 --2- 172.21.15.38:0/448214528 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1674060890 0x7f1674062d40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1694156aa0 tx=0x7f1684015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:59:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.273+0000 7f1690ff9700 1 -- 172.21.15.38:0/448214528 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f168c051570 con 0x7f16940aa600 2024-08-15T17:59:37.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T17:59:37.465+0000 7f1699372700 1 -- 172.21.15.38:0/448214528 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000845064", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f169414b9c0 con 0x7f1674060890 2024-08-15T18:00:05.334 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:00:05.338 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:00:35.388 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:00:35.392 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:01:05.444 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:01:05.447 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:01:35.520 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:01:35.524 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:01:37.161 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-08-15T18:01:37.162 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-08-15T18:01:37.162 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T18:01:37.183 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T18:01:37.203 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-08-15T18:01:37.203 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-08-15T18:01:37.203 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T18:01:37.203 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-08-15T18:01:37.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[33069]: fuse finished with error 0 and tester_r 0 2024-08-15T18:01:37.219 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T18:01:37.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:37.220 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-15T18:01:37.382 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2024-08-15T18:01:37.382 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T18:01:37.383 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-08-15T18:01:37.398 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi104.stderr:ceph-fuse[30232]: fuse finished with error 0 and tester_r 0 2024-08-15T18:01:37.400 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T18:01:37.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:37.401 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-08-15T18:01:37.525 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T18:01:37.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:37.525 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-08-15T18:01:37.533 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-08-15T18:01:37.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T18:01:37.534 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T18:01:37.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:37.534 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-08-15T18:01:37.541 INFO:teuthology.orchestra.run.smithi104.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-08-15T18:01:37.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T18:01:37.542 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T18:01:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb8a59700 1 Processor -- start 2024-08-15T18:01:37.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb8a59700 1 -- start start 2024-08-15T18:01:37.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb8a59700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7fb4155390 0x7f7fb4155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:37.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb8a59700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7fb40c9dd0 0x7f7fb4068b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:37.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb8a59700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb406b6d0 con 0x7f7fb40682a0 2024-08-15T18:01:37.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb8a59700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb4068700 con 0x7f7fb4155390 2024-08-15T18:01:37.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb8a59700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb406b3f0 con 0x7f7fb40c9dd0 2024-08-15T18:01:37.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb2d9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7fb4155390 0x7f7fb4155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:37.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb2d9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7fb4155390 0x7f7fb4155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43866/0 (socket says 172.21.15.38:43866) 2024-08-15T18:01:37.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb2d9d700 1 -- 172.21.15.38:0/1373761321 learned_addr learned my addr 172.21.15.38:0/1373761321 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:37.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb359e700 1 --2- 172.21.15.38:0/1373761321 >> v2:172.21.15.104:3300/0 conn(0x7f7fb40c9dd0 0x7f7fb4068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:37.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb2d9d700 1 -- 172.21.15.38:0/1373761321 >> v2:172.21.15.104:3300/0 conn(0x7f7fb40c9dd0 msgr2=0x7f7fb4068b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:37.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb2d9d700 1 --2- 172.21.15.38:0/1373761321 >> v2:172.21.15.104:3300/0 conn(0x7f7fb40c9dd0 0x7f7fb4068b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:37.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb2d9d700 1 -- 172.21.15.38:0/1373761321 >> v1:172.21.15.104:6789/0 conn(0x7f7fb40682a0 legacy=0x7f7fb40c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:37.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.690+0000 7f7fb2d9d700 1 -- 172.21.15.38:0/1373761321 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40677c0 con 0x7f7fb4155390 2024-08-15T18:01:37.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb2d9d700 1 --2- 172.21.15.38:0/1373761321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fb4155390 0x7f7fb4155760 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40cc4e0 tx=0x7f7fa4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91525e4ba81244ee server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:37.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb1d9b700 1 -- 172.21.15.38:0/1373761321 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4009e70 con 0x7f7fb4155390 2024-08-15T18:01:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb1d9b700 1 -- 172.21.15.38:0/1373761321 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa400b860 con 0x7f7fb4155390 2024-08-15T18:01:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb1d9b700 1 -- 172.21.15.38:0/1373761321 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400bc00 con 0x7f7fb4155390 2024-08-15T18:01:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb8a59700 1 -- 172.21.15.38:0/1373761321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fb4155390 msgr2=0x7f7fb4155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb8a59700 1 --2- 172.21.15.38:0/1373761321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fb4155390 0x7f7fb4155760 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40cc4e0 tx=0x7f7fa4009250 comp rx=0 tx=0).stop 2024-08-15T18:01:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb8a59700 1 -- 172.21.15.38:0/1373761321 shutdown_connections 2024-08-15T18:01:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb8a59700 1 --2- 172.21.15.38:0/1373761321 >> v2:172.21.15.104:3300/0 conn(0x7f7fb40c9dd0 0x7f7fb4068b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb8a59700 1 --2- 172.21.15.38:0/1373761321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fb4155390 0x7f7fb4155760 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb8a59700 1 -- 172.21.15.38:0/1373761321 >> 172.21.15.38:0/1373761321 conn(0x7f7fb40ccfc0 msgr2=0x7f7fb40c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb8a59700 1 -- 172.21.15.38:0/1373761321 shutdown_connections 2024-08-15T18:01:37.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb8a59700 1 -- 172.21.15.38:0/1373761321 wait complete. 2024-08-15T18:01:37.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb8a59700 1 Processor -- start 2024-08-15T18:01:37.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.694+0000 7f7fb8a59700 1 -- start start 2024-08-15T18:01:37.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb8a59700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fb40682a0 0x7f7fb414e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:37.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb8a59700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb40c9dd0 0x7f7fb4156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:37.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb8a59700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7fb4155390 0x7f7fb4153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb8a59700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40c8820 con 0x7f7fb40682a0 2024-08-15T18:01:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb8a59700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40c7c10 con 0x7f7fb40c9dd0 2024-08-15T18:01:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb8a59700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40c7ef0 con 0x7f7fb4155390 2024-08-15T18:01:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fb40682a0 0x7f7fb414e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fb40682a0 0x7f7fb414e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43878/0 (socket says 172.21.15.38:43878) 2024-08-15T18:01:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb2d9d700 1 -- 172.21.15.38:0/561826657 learned_addr learned my addr 172.21.15.38:0/561826657 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb259c700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb40c9dd0 0x7f7fb4156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb359e700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7fb4155390 0x7f7fb4153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb2d9d700 1 -- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7fb4155390 msgr2=0x7f7fb4153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb2d9d700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7fb4155390 0x7f7fb4153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb2d9d700 1 -- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb40c9dd0 msgr2=0x7f7fb4156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb2d9d700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb40c9dd0 0x7f7fb4156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb2d9d700 1 -- 172.21.15.38:0/561826657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40094f0 con 0x7f7fb40682a0 2024-08-15T18:01:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb359e700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7fb4155390 0x7f7fb4153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb2d9d700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fb40682a0 0x7f7fb414e9e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4069330 tx=0x7f7fa4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7f9affd700 1 -- 172.21.15.38:0/561826657 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400bbe0 con 0x7f7fb40682a0 2024-08-15T18:01:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7f9affd700 1 -- 172.21.15.38:0/561826657 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa4011420 con 0x7f7fb40682a0 2024-08-15T18:01:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb8a59700 1 -- 172.21.15.38:0/561826657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb40c81d0 con 0x7f7fb40682a0 2024-08-15T18:01:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7f9affd700 1 -- 172.21.15.38:0/561826657 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40117a0 con 0x7f7fb40682a0 2024-08-15T18:01:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb8a59700 1 -- 172.21.15.38:0/561826657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb414ef20 con 0x7f7fb40682a0 2024-08-15T18:01:37.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.698+0000 7f7fb8a59700 1 -- 172.21.15.38:0/561826657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb40638f0 con 0x7f7fb40682a0 2024-08-15T18:01:37.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.702+0000 7f7f9affd700 1 -- 172.21.15.38:0/561826657 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7fa4011b90 con 0x7f7fb40682a0 2024-08-15T18:01:37.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.702+0000 7f7f9affd700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7fa0060860 0x7f7fa0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:37.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.702+0000 7f7f9affd700 1 -- 172.21.15.38:0/561826657 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f7fa407e7c0 con 0x7f7fb40682a0 2024-08-15T18:01:37.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.702+0000 7f7fb259c700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7fa0060860 0x7f7fa0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:37.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.702+0000 7f7fb259c700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7fa0060860 0x7f7fa0062d10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4155b80 tx=0x7f7fa8007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.710+0000 7f7f9affd700 1 -- 172.21.15.38:0/561826657 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7fa404e890 con 0x7f7fb40682a0 2024-08-15T18:01:37.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.910+0000 7f7fb8a59700 1 -- 172.21.15.38:0/561826657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fb406ea50 con 0x7f7fb40682a0 2024-08-15T18:01:37.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.910+0000 7f7f9affd700 1 -- 172.21.15.38:0/561826657 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5681 (secure 0 0 0) 0x7f7fa4051eb0 con 0x7f7fb40682a0 2024-08-15T18:01:37.914 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:37.914 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/373633111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":373633111},{"type":"v1","addr":"172.21.15.38:6833","nonce":373633111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/389886189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":389886189},{"type":"v1","addr":"172.21.15.38:6835","nonce":389886189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:20.353438+0000","modified":"2024-08-15T17:59:24.015140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4999},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4999":{"gid":4999,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/1865762791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1865762791},{"type":"v1","addr":"172.21.15.104:6837","nonce":1865762791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:35.121185+0000","modified":"2024-08-15T17:59:36.892161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4981":{"gid":4981,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6835/875828080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":875828080},{"type":"v1","addr":"172.21.15.104:6835","nonce":875828080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000845064","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T18:01:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 -- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7fa0060860 msgr2=0x7f7fa0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7fa0060860 0x7f7fa0062d10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4155b80 tx=0x7f7fa8007ec0 comp rx=0 tx=0).stop 2024-08-15T18:01:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 -- 172.21.15.38:0/561826657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fb40682a0 msgr2=0x7f7fb414e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fb40682a0 0x7f7fb414e9e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4069330 tx=0x7f7fa4017000 comp rx=0 tx=0).stop 2024-08-15T18:01:37.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 -- 172.21.15.38:0/561826657 shutdown_connections 2024-08-15T18:01:37.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7fa0060860 0x7f7fa0062d10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:37.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7fb4155390 0x7f7fb4153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:37.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb40c9dd0 0x7f7fb4156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:37.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 --2- 172.21.15.38:0/561826657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fb40682a0 0x7f7fb414e9e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:37.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 -- 172.21.15.38:0/561826657 >> 172.21.15.38:0/561826657 conn(0x7f7fb40ccfc0 msgr2=0x7f7fb40cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:37.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 -- 172.21.15.38:0/561826657 shutdown_connections 2024-08-15T18:01:37.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:37.914+0000 7f7fb8a59700 1 -- 172.21.15.38:0/561826657 wait complete. 2024-08-15T18:01:37.918 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 18 2024-08-15T18:01:37.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T18:01:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd389db8700 1 Processor -- start 2024-08-15T18:01:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd389db8700 1 -- start start 2024-08-15T18:01:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd389db8700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd384159040 0x7fd3840cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd389db8700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd3840d0010 0x7fd3840d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd389db8700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3840cf190 con 0x7fd38415cea0 2024-08-15T18:01:38.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd389db8700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3840d09a0 con 0x7fd3840d0010 2024-08-15T18:01:38.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd389db8700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840ce5e0 con 0x7fd384159040 2024-08-15T18:01:38.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd383fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd3840d0010 0x7fd3840d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd383fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd3840d0010 0x7fd3840d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43880/0 (socket says 172.21.15.38:43880) 2024-08-15T18:01:38.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd388db6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd384159040 0x7fd3840cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd383fff700 1 -- 172.21.15.38:0/3241772362 learned_addr learned my addr 172.21.15.38:0/3241772362 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd383fff700 1 -- 172.21.15.38:0/3241772362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd384159040 msgr2=0x7fd3840cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd383fff700 1 --2- 172.21.15.38:0/3241772362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd384159040 0x7fd3840cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd383fff700 1 -- 172.21.15.38:0/3241772362 >> v1:172.21.15.104:6789/0 conn(0x7fd38415cea0 legacy=0x7fd38415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd383fff700 1 -- 172.21.15.38:0/3241772362 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840ce860 con 0x7fd3840d0010 2024-08-15T18:01:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd383fff700 1 --2- 172.21.15.38:0/3241772362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3840d0010 0x7fd3840d0460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0099b0 tx=0x7fd36c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aaa49d459037f1e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd3837fe700 1 -- 172.21.15.38:0/3241772362 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c009ea0 con 0x7fd3840d0010 2024-08-15T18:01:38.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd3837fe700 1 -- 172.21.15.38:0/3241772362 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c00b860 con 0x7fd3840d0010 2024-08-15T18:01:38.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.034+0000 7fd3837fe700 1 -- 172.21.15.38:0/3241772362 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00bc00 con 0x7fd3840d0010 2024-08-15T18:01:38.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.038+0000 7fd389db8700 1 -- 172.21.15.38:0/3241772362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3840d0010 msgr2=0x7fd3840d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.038+0000 7fd389db8700 1 --2- 172.21.15.38:0/3241772362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3840d0010 0x7fd3840d0460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0099b0 tx=0x7fd36c009250 comp rx=0 tx=0).stop 2024-08-15T18:01:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.038+0000 7fd389db8700 1 -- 172.21.15.38:0/3241772362 shutdown_connections 2024-08-15T18:01:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.038+0000 7fd389db8700 1 --2- 172.21.15.38:0/3241772362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3840d0010 0x7fd3840d0460 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.038+0000 7fd389db8700 1 --2- 172.21.15.38:0/3241772362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd384159040 0x7fd3840cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.038+0000 7fd389db8700 1 -- 172.21.15.38:0/3241772362 >> 172.21.15.38:0/3241772362 conn(0x7fd384068250 msgr2=0x7fd384068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.038+0000 7fd389db8700 1 -- 172.21.15.38:0/3241772362 shutdown_connections 2024-08-15T18:01:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.038+0000 7fd389db8700 1 -- 172.21.15.38:0/3241772362 wait complete. 2024-08-15T18:01:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.038+0000 7fd389db8700 1 Processor -- start 2024-08-15T18:01:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.038+0000 7fd389db8700 1 -- start start 2024-08-15T18:01:38.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd389db8700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3840d0010 0x7fd38416c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd389db8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd384159040 0x7fd3840a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd389db8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38415cea0 0x7fd3840a7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd389db8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840a8ab0 con 0x7fd38415cea0 2024-08-15T18:01:38.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd389db8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd384169880 con 0x7fd384159040 2024-08-15T18:01:38.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd389db8700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840b17a0 con 0x7fd3840d0010 2024-08-15T18:01:38.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd3895b7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38415cea0 0x7fd3840a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd388db6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3840d0010 0x7fd38416c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd383fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd384159040 0x7fd3840a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd383fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd384159040 0x7fd3840a73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:56256/0 (socket says 172.21.15.38:56256) 2024-08-15T18:01:38.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd388db6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3840d0010 0x7fd38416c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:59044/0 (socket says 172.21.15.38:59044) 2024-08-15T18:01:38.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd388db6700 1 -- 172.21.15.38:0/3828376302 learned_addr learned my addr 172.21.15.38:0/3828376302 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:38.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd3895b7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38415cea0 0x7fd3840a7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43894/0 (socket says 172.21.15.38:43894) 2024-08-15T18:01:38.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd3895b7700 1 -- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3840d0010 msgr2=0x7fd38416c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd3895b7700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3840d0010 0x7fd38416c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd3895b7700 1 -- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd384159040 msgr2=0x7fd3840a73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd3895b7700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd384159040 0x7fd3840a73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd3895b7700 1 -- 172.21.15.38:0/3828376302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c0094f0 con 0x7fd38415cea0 2024-08-15T18:01:38.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd3895b7700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38415cea0 0x7fd3840a7900 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd37c001eb0 tx=0x7fd37c007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd380ff9700 1 -- 172.21.15.38:0/3828376302 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c00ce10 con 0x7fd38415cea0 2024-08-15T18:01:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd380ff9700 1 -- 172.21.15.38:0/3828376302 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c011590 con 0x7fd38415cea0 2024-08-15T18:01:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd389db8700 1 -- 172.21.15.38:0/3828376302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3840a7f30 con 0x7fd38415cea0 2024-08-15T18:01:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd380ff9700 1 -- 172.21.15.38:0/3828376302 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c011950 con 0x7fd38415cea0 2024-08-15T18:01:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.042+0000 7fd389db8700 1 -- 172.21.15.38:0/3828376302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3840a84f0 con 0x7fd38415cea0 2024-08-15T18:01:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.046+0000 7fd389db8700 1 -- 172.21.15.38:0/3828376302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd384169a60 con 0x7fd38415cea0 2024-08-15T18:01:38.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.046+0000 7fd380ff9700 1 -- 172.21.15.38:0/3828376302 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd37c011d00 con 0x7fd38415cea0 2024-08-15T18:01:38.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.046+0000 7fd380ff9700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd364060860 0x7fd364062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.046+0000 7fd380ff9700 1 -- 172.21.15.38:0/3828376302 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fd37c07c200 con 0x7fd38415cea0 2024-08-15T18:01:38.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.046+0000 7fd388db6700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd364060860 0x7fd364062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.046+0000 7fd388db6700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd364060860 0x7fd364062d10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd374009fd0 tx=0x7fd37400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:38.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.058+0000 7fd380ff9700 1 -- 172.21.15.38:0/3828376302 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd384169a60 con 0x7fd38415cea0 2024-08-15T18:01:38.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.222+0000 7fd389db8700 1 -- 172.21.15.38:0/3828376302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd384169a60 con 0x7fd38415cea0 2024-08-15T18:01:38.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.222+0000 7fd380ff9700 1 -- 172.21.15.38:0/3828376302 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18599 (secure 0 0 0) 0x7fd384169a60 con 0x7fd38415cea0 2024-08-15T18:01:38.226 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:38.226 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":37,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:59:38.114973+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:59:18.793510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:59:19.328139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000845064.meta","create_time":"2024-08-15T17:59:33.132275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000845064"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000845064.data","create_time":"2024-08-15T17:59:34.143101+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000845064"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T18:01:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 -- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd364060860 msgr2=0x7fd364062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd364060860 0x7fd364062d10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd374009fd0 tx=0x7fd37400b040 comp rx=0 tx=0).stop 2024-08-15T18:01:38.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 -- 172.21.15.38:0/3828376302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38415cea0 msgr2=0x7fd3840a7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38415cea0 0x7fd3840a7900 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd37c001eb0 tx=0x7fd37c007e20 comp rx=0 tx=0).stop 2024-08-15T18:01:38.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 -- 172.21.15.38:0/3828376302 shutdown_connections 2024-08-15T18:01:38.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fd364060860 0x7fd364062d10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38415cea0 0x7fd3840a7900 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd384159040 0x7fd3840a73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 --2- 172.21.15.38:0/3828376302 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3840d0010 0x7fd38416c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 -- 172.21.15.38:0/3828376302 >> 172.21.15.38:0/3828376302 conn(0x7fd384068250 msgr2=0x7fd384156e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:38.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 -- 172.21.15.38:0/3828376302 shutdown_connections 2024-08-15T18:01:38.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.226+0000 7fd389db8700 1 -- 172.21.15.38:0/3828376302 wait complete. 2024-08-15T18:01:38.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:59:18.793510+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:59:19.328139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs.volume_0000000000845064.meta', 'create_time': '2024-08-15T17:59:33.132275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000845064'}}}, {'pool': 7, 'pool_name': 'cephfs.volume_0000000000845064.data', 'create_time': '2024-08-15T17:59:34.143101+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000845064'}}}] 2024-08-15T18:01:38.241 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T18:01:38.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f32097d1700 1 Processor -- start 2024-08-15T18:01:38.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f32097d1700 1 -- start start 2024-08-15T18:01:38.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f32097d1700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3204159400 0x7f32040b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f32097d1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f320406bac0 0x7f32040686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f32097d1700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32040adb00 con 0x7f32041597d0 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f32097d1700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320406bf40 con 0x7f320406bac0 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f32097d1700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32040ac630 con 0x7f3204159400 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f31fa7fc700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f32041597d0 0x7f32040ad3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50482/0 (socket says 172.21.15.38:50482) 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f31fa7fc700 1 -- 172.21.15.38:0/2494011789 learned_addr learned my addr 172.21.15.38:0/2494011789 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f3202ffd700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3204159400 0x7f32040b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f32037fe700 1 --2- 172.21.15.38:0/2494011789 >> v2:172.21.15.104:3300/0 conn(0x7f320406bac0 0x7f32040686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243295543 0 0) 0x7f32040adb00 con 0x7f32041597d0 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.358+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31f0003570 con 0x7f32041597d0 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445079475 0 0) 0x7f31f0003570 con 0x7f32041597d0 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3204159400 msgr2=0x7f32040b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3204159400 0x7f32040b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 >> v2:172.21.15.104:3300/0 conn(0x7f320406bac0 msgr2=0x7f32040686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:38.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 --2- 172.21.15.38:0/2494011789 >> v2:172.21.15.104:3300/0 conn(0x7f320406bac0 0x7f32040686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32040ac910 con 0x7f32041597d0 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f31ec001470 con 0x7f32041597d0 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 >> v1:172.21.15.38:6789/0 conn(0x7f32041597d0 legacy=0x7f32040ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3204159400 0x7f31f0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f0003fe0 0x7f31f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31f0004d50 0x7f31f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320406bf40 con 0x7f31f0003fe0 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32040ac630 con 0x7f31f0004d50 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f000ba90 con 0x7f3204159400 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f31fa7fc700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f0003fe0 0x7f31f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f3202ffd700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3204159400 0x7f31f0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32037fe700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31f0004d50 0x7f31f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f31fa7fc700 1 -- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3204159400 msgr2=0x7f31f0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:38.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f31fa7fc700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3204159400 0x7f31f0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f31fa7fc700 1 -- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31f0004d50 msgr2=0x7f31f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:38.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f31fa7fc700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31f0004d50 0x7f31f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f31fa7fc700 1 -- 172.21.15.38:0/2494011789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f0003570 con 0x7f31f0003fe0 2024-08-15T18:01:38.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32037fe700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31f0004d50 0x7f31f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T18:01:38.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f31fa7fc700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f0003fe0 0x7f31f0004810 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c390 tx=0x7f32040b2540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5490a7157751aa0e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:38.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00cc00 con 0x7f31f0003fe0 2024-08-15T18:01:38.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec00cd60 con 0x7f31f0003fe0 2024-08-15T18:01:38.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32027fc700 1 -- 172.21.15.38:0/2494011789 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec005b70 con 0x7f31f0003fe0 2024-08-15T18:01:38.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32097d1700 1 -- 172.21.15.38:0/2494011789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f0003fe0 msgr2=0x7f31f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32097d1700 1 --2- 172.21.15.38:0/2494011789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f0003fe0 0x7f31f0004810 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c390 tx=0x7f32040b2540 comp rx=0 tx=0).stop 2024-08-15T18:01:38.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32037fe700 1 -- 172.21.15.38:0/2494011789 reap_dead start 2024-08-15T18:01:38.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32097d1700 1 -- 172.21.15.38:0/2494011789 shutdown_connections 2024-08-15T18:01:38.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32097d1700 1 -- 172.21.15.38:0/2494011789 >> 172.21.15.38:0/2494011789 conn(0x7f32040c9bf0 msgr2=0x7f320406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:38.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32097d1700 1 -- 172.21.15.38:0/2494011789 shutdown_connections 2024-08-15T18:01:38.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32097d1700 1 -- 172.21.15.38:0/2494011789 wait complete. 2024-08-15T18:01:38.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32097d1700 1 Processor -- start 2024-08-15T18:01:38.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.362+0000 7f32097d1700 1 -- start start 2024-08-15T18:01:38.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32097d1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31f0003fe0 0x7f32040c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32097d1700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32040b18e0 0x7f320414d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32097d1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32040b0610 0x7f32040b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32097d1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32040c2d60 con 0x7f32040b0610 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32097d1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32040b0fc0 con 0x7f31f0003fe0 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32097d1700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32040c28d0 con 0x7f32040b18e0 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32037fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32040b0610 0x7f32040b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f3202ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31f0003fe0 0x7f32040c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32037fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32040b0610 0x7f32040b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43908/0 (socket says 172.21.15.38:43908) 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32037fe700 1 -- 172.21.15.38:0/945259099 learned_addr learned my addr 172.21.15.38:0/945259099 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f31fa7fc700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32040b18e0 0x7f320414d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32037fe700 1 -- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32040b18e0 msgr2=0x7f320414d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32037fe700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32040b18e0 0x7f320414d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32037fe700 1 -- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31f0003fe0 msgr2=0x7f32040c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32037fe700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31f0003fe0 0x7f32040c15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32037fe700 1 -- 172.21.15.38:0/945259099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32040b1d50 con 0x7f32040b0610 2024-08-15T18:01:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f31fa7fc700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32040b18e0 0x7f320414d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32037fe700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32040b0610 0x7f32040b0a80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3204070500 tx=0x7f31fc00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f31fb7fe700 1 -- 172.21.15.38:0/945259099 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc008a40 con 0x7f32040b0610 2024-08-15T18:01:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f31fb7fe700 1 -- 172.21.15.38:0/945259099 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc011040 con 0x7f32040b0610 2024-08-15T18:01:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32097d1700 1 -- 172.21.15.38:0/945259099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32040c1cc0 con 0x7f32040b0610 2024-08-15T18:01:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f31fb7fe700 1 -- 172.21.15.38:0/945259099 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc008dd0 con 0x7f32040b0610 2024-08-15T18:01:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32097d1700 1 -- 172.21.15.38:0/945259099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32040c2280 con 0x7f32040b0610 2024-08-15T18:01:38.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.366+0000 7f32097d1700 1 -- 172.21.15.38:0/945259099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32040c2a60 con 0x7f32040b0610 2024-08-15T18:01:38.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.370+0000 7f31fb7fe700 1 -- 172.21.15.38:0/945259099 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f31fc017610 con 0x7f32040b0610 2024-08-15T18:01:38.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.370+0000 7f31fb7fe700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f31e4060860 0x7f31e4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.370+0000 7f31fb7fe700 1 -- 172.21.15.38:0/945259099 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f31fc07d1e0 con 0x7f32040b0610 2024-08-15T18:01:38.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.370+0000 7f3202ffd700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f31e4060860 0x7f31e4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.370+0000 7f3202ffd700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f31e4060860 0x7f31e4062d10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3204067cf0 tx=0x7f31f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:38.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.378+0000 7f31fb7fe700 1 -- 172.21.15.38:0/945259099 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f31fc04d2b0 con 0x7f32040b0610 2024-08-15T18:01:38.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.546+0000 7f32097d1700 1 -- 172.21.15.38:0/945259099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32040645c0 con 0x7f32040b0610 2024-08-15T18:01:38.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.546+0000 7f31fb7fe700 1 -- 172.21.15.38:0/945259099 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31fc0508d0 con 0x7f32040b0610 2024-08-15T18:01:38.549 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T18:01:38.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 -- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f31e4060860 msgr2=0x7f31e4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f31e4060860 0x7f31e4062d10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3204067cf0 tx=0x7f31f4008040 comp rx=0 tx=0).stop 2024-08-15T18:01:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 -- 172.21.15.38:0/945259099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32040b0610 msgr2=0x7f32040b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32040b0610 0x7f32040b0a80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3204070500 tx=0x7f31fc00d040 comp rx=0 tx=0).stop 2024-08-15T18:01:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 -- 172.21.15.38:0/945259099 shutdown_connections 2024-08-15T18:01:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f31e4060860 0x7f31e4062d10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32040b0610 0x7f32040b0a80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32040b18e0 0x7f320414d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 --2- 172.21.15.38:0/945259099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31f0003fe0 0x7f32040c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 -- 172.21.15.38:0/945259099 >> 172.21.15.38:0/945259099 conn(0x7f32040c9bf0 msgr2=0x7f32040ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 -- 172.21.15.38:0/945259099 shutdown_connections 2024-08-15T18:01:38.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.550+0000 7f32097d1700 1 -- 172.21.15.38:0/945259099 wait complete. 2024-08-15T18:01:38.569 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T18:01:38.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.678+0000 7f53e8cb7700 1 Processor -- start 2024-08-15T18:01:38.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.678+0000 7f53e8cb7700 1 -- start start 2024-08-15T18:01:38.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e8cb7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f53e4153a30 0x7f53e40ae500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e8cb7700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f53e415a680 0x7f53e406c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e8cb7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53e41570b0 0x7f53e406c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e8cb7700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40af0c0 con 0x7f53e415a680 2024-08-15T18:01:38.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e8cb7700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e415a1f0 con 0x7f53e4153a30 2024-08-15T18:01:38.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e8cb7700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4153e00 con 0x7f53e41570b0 2024-08-15T18:01:38.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e2ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f53e415a680 0x7f53e406c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e2ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f53e415a680 0x7f53e406c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43912/0 (socket says 172.21.15.38:43912) 2024-08-15T18:01:38.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e2ffd700 1 -- 172.21.15.38:0/3503350307 learned_addr learned my addr 172.21.15.38:0/3503350307 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e37fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f53e4153a30 0x7f53e40ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e3fff700 1 --2- 172.21.15.38:0/3503350307 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53e41570b0 0x7f53e406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e2ffd700 1 -- 172.21.15.38:0/3503350307 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53e41570b0 msgr2=0x7f53e406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e2ffd700 1 --2- 172.21.15.38:0/3503350307 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53e41570b0 0x7f53e406c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e2ffd700 1 -- 172.21.15.38:0/3503350307 >> v2:172.21.15.104:3300/0 conn(0x7f53e4153a30 msgr2=0x7f53e40ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e2ffd700 1 --2- 172.21.15.38:0/3503350307 >> v2:172.21.15.104:3300/0 conn(0x7f53e4153a30 0x7f53e40ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e2ffd700 1 -- 172.21.15.38:0/3503350307 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e40aede0 con 0x7f53e415a680 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e37fe700 1 --2- 172.21.15.38:0/3503350307 >> v2:172.21.15.104:3300/0 conn(0x7f53e4153a30 0x7f53e40ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e2ffd700 1 --2- 172.21.15.38:0/3503350307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e415a680 0x7f53e406c200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f53e4064cb0 tx=0x7f53d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23de45dc9c00af54 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e27fc700 1 -- 172.21.15.38:0/3503350307 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400b640 con 0x7f53e415a680 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e27fc700 1 -- 172.21.15.38:0/3503350307 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d400b7a0 con 0x7f53e415a680 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.682+0000 7f53e27fc700 1 -- 172.21.15.38:0/3503350307 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400bb60 con 0x7f53e415a680 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.686+0000 7f53e8cb7700 1 -- 172.21.15.38:0/3503350307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e415a680 msgr2=0x7f53e406c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.686+0000 7f53e8cb7700 1 --2- 172.21.15.38:0/3503350307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e415a680 0x7f53e406c200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f53e4064cb0 tx=0x7f53d4009250 comp rx=0 tx=0).stop 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.686+0000 7f53e8cb7700 1 -- 172.21.15.38:0/3503350307 shutdown_connections 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.686+0000 7f53e8cb7700 1 --2- 172.21.15.38:0/3503350307 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53e41570b0 0x7f53e406c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.686+0000 7f53e8cb7700 1 --2- 172.21.15.38:0/3503350307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e415a680 0x7f53e406c200 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.686+0000 7f53e8cb7700 1 --2- 172.21.15.38:0/3503350307 >> v2:172.21.15.104:3300/0 conn(0x7f53e4153a30 0x7f53e40ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.686+0000 7f53e8cb7700 1 -- 172.21.15.38:0/3503350307 >> 172.21.15.38:0/3503350307 conn(0x7f53e40ca2b0 msgr2=0x7f53e406baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:38.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.686+0000 7f53e8cb7700 1 -- 172.21.15.38:0/3503350307 shutdown_connections 2024-08-15T18:01:38.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.686+0000 7f53e8cb7700 1 -- 172.21.15.38:0/3503350307 wait complete. 2024-08-15T18:01:38.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.686+0000 7f53e8cb7700 1 Processor -- start 2024-08-15T18:01:38.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.686+0000 7f53e8cb7700 1 -- start start 2024-08-15T18:01:38.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e8cb7700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53e4153a30 0x7f53e4151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e8cb7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e41570b0 0x7f53e40b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e8cb7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53e415a680 0x7f53e41651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e8cb7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4151460 con 0x7f53e41570b0 2024-08-15T18:01:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e8cb7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4151c90 con 0x7f53e415a680 2024-08-15T18:01:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e8cb7700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4067b90 con 0x7f53e4153a30 2024-08-15T18:01:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e41570b0 0x7f53e40b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e41570b0 0x7f53e40b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43926/0 (socket says 172.21.15.38:43926) 2024-08-15T18:01:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e2ffd700 1 -- 172.21.15.38:0/4052356017 learned_addr learned my addr 172.21.15.38:0/4052356017 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e37fe700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53e4153a30 0x7f53e4151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e3fff700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53e415a680 0x7f53e41651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e2ffd700 1 -- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53e4153a30 msgr2=0x7f53e4151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e2ffd700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53e4153a30 0x7f53e4151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e2ffd700 1 -- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53e415a680 msgr2=0x7f53e41651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e2ffd700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53e415a680 0x7f53e41651f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e2ffd700 1 -- 172.21.15.38:0/4052356017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc009a20 con 0x7f53e41570b0 2024-08-15T18:01:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e2ffd700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e41570b0 0x7f53e40b2150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f53e406dde0 tx=0x7f53d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53cb7fe700 1 -- 172.21.15.38:0/4052356017 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400b910 con 0x7f53e41570b0 2024-08-15T18:01:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53cb7fe700 1 -- 172.21.15.38:0/4052356017 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d400ba70 con 0x7f53e41570b0 2024-08-15T18:01:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53cb7fe700 1 -- 172.21.15.38:0/4052356017 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d401a660 con 0x7f53e41570b0 2024-08-15T18:01:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e8cb7700 1 -- 172.21.15.38:0/4052356017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d40094f0 con 0x7f53e41570b0 2024-08-15T18:01:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e8cb7700 1 -- 172.21.15.38:0/4052356017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e4068150 con 0x7f53e41570b0 2024-08-15T18:01:38.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.690+0000 7f53e8cb7700 1 -- 172.21.15.38:0/4052356017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e40638f0 con 0x7f53e41570b0 2024-08-15T18:01:38.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.698+0000 7f53cb7fe700 1 -- 172.21.15.38:0/4052356017 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f53d401aa50 con 0x7f53e41570b0 2024-08-15T18:01:38.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.698+0000 7f53cb7fe700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f53d0060860 0x7f53d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.698+0000 7f53cb7fe700 1 -- 172.21.15.38:0/4052356017 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f53d407ef60 con 0x7f53e41570b0 2024-08-15T18:01:38.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.698+0000 7f53e37fe700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f53d0060860 0x7f53d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.698+0000 7f53e37fe700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f53d0060860 0x7f53d0062d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009fd0 tx=0x7f53dc00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:38.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.702+0000 7f53cb7fe700 1 -- 172.21.15.38:0/4052356017 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f53d404f010 con 0x7f53e41570b0 2024-08-15T18:01:38.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.862+0000 7f53e8cb7700 1 -- 172.21.15.38:0/4052356017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53e40c95a0 con 0x7f53e41570b0 2024-08-15T18:01:38.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.862+0000 7f53cb7fe700 1 -- 172.21.15.38:0/4052356017 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f53d4052630 con 0x7f53e41570b0 2024-08-15T18:01:38.866 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T18:01:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 -- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f53d0060860 msgr2=0x7f53d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f53d0060860 0x7f53d0062d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009fd0 tx=0x7f53dc00b040 comp rx=0 tx=0).stop 2024-08-15T18:01:38.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 -- 172.21.15.38:0/4052356017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e41570b0 msgr2=0x7f53e40b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e41570b0 0x7f53e40b2150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f53e406dde0 tx=0x7f53d4017000 comp rx=0 tx=0).stop 2024-08-15T18:01:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 -- 172.21.15.38:0/4052356017 shutdown_connections 2024-08-15T18:01:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f53d0060860 0x7f53d0062d10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53e415a680 0x7f53e41651f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e41570b0 0x7f53e40b2150 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 --2- 172.21.15.38:0/4052356017 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53e4153a30 0x7f53e4151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 -- 172.21.15.38:0/4052356017 >> 172.21.15.38:0/4052356017 conn(0x7f53e40ca2b0 msgr2=0x7f53e414f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 -- 172.21.15.38:0/4052356017 shutdown_connections 2024-08-15T18:01:38.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.866+0000 7f53e8cb7700 1 -- 172.21.15.38:0/4052356017 wait complete. 2024-08-15T18:01:38.880 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T18:01:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.994+0000 7f61bab22700 1 Processor -- start 2024-08-15T18:01:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.994+0000 7f61bab22700 1 -- start start 2024-08-15T18:01:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.994+0000 7f61bab22700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f61b41512e0 0x7f61b41516b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.994+0000 7f61bab22700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b415b280 0x7f61b4160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b40cd3d0 con 0x7f61b40c8540 2024-08-15T18:01:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b40c4d80 con 0x7f61b41512e0 2024-08-15T18:01:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40cc4e0 con 0x7f61b415b280 2024-08-15T18:01:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61b9b20700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f61b41512e0 0x7f61b41516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61b9b20700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f61b41512e0 0x7f61b41516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43942/0 (socket says 172.21.15.38:43942) 2024-08-15T18:01:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61b9b20700 1 -- 172.21.15.38:0/3933347210 learned_addr learned my addr 172.21.15.38:0/3933347210 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:39.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61ba321700 1 --2- 172.21.15.38:0/3933347210 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b415b280 0x7f61b4160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61b9b20700 1 -- 172.21.15.38:0/3933347210 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b415b280 msgr2=0x7f61b4160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:39.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61b9b20700 1 --2- 172.21.15.38:0/3933347210 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b415b280 0x7f61b4160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61b9b20700 1 -- 172.21.15.38:0/3933347210 >> v1:172.21.15.104:6789/0 conn(0x7f61b40c8540 legacy=0x7f61b415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61b9b20700 1 -- 172.21.15.38:0/3933347210 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b40cc7c0 con 0x7f61b41512e0 2024-08-15T18:01:39.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61b9b20700 1 --2- 172.21.15.38:0/3933347210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b41512e0 0x7f61b41516b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009f20 tx=0x7f61b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ebb2b480976c77 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61b8b1e700 1 -- 172.21.15.38:0/3933347210 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b000e070 con 0x7f61b41512e0 2024-08-15T18:01:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61b8b1e700 1 -- 172.21.15.38:0/3933347210 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b000b980 con 0x7f61b41512e0 2024-08-15T18:01:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61b8b1e700 1 -- 172.21.15.38:0/3933347210 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00092e0 con 0x7f61b41512e0 2024-08-15T18:01:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 -- 172.21.15.38:0/3933347210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b41512e0 msgr2=0x7f61b41516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 --2- 172.21.15.38:0/3933347210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b41512e0 0x7f61b41516b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009f20 tx=0x7f61b0009840 comp rx=0 tx=0).stop 2024-08-15T18:01:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 -- 172.21.15.38:0/3933347210 shutdown_connections 2024-08-15T18:01:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 --2- 172.21.15.38:0/3933347210 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b415b280 0x7f61b4160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 --2- 172.21.15.38:0/3933347210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b41512e0 0x7f61b41516b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 -- 172.21.15.38:0/3933347210 >> 172.21.15.38:0/3933347210 conn(0x7f61b40cdf00 msgr2=0x7f61b40ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 -- 172.21.15.38:0/3933347210 shutdown_connections 2024-08-15T18:01:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 -- 172.21.15.38:0/3933347210 wait complete. 2024-08-15T18:01:39.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 Processor -- start 2024-08-15T18:01:39.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 -- start start 2024-08-15T18:01:39.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:38.998+0000 7f61bab22700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b40c8540 0x7f61b4069cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61bab22700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b41512e0 0x7f61b40c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61bab22700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b415b280 0x7f61b40c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61bab22700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b4069250 con 0x7f61b41512e0 2024-08-15T18:01:39.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61bab22700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b406a200 con 0x7f61b415b280 2024-08-15T18:01:39.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61bab22700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40c7a50 con 0x7f61b40c8540 2024-08-15T18:01:39.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61b931f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b41512e0 0x7f61b40c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61b9b20700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b40c8540 0x7f61b4069cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61b931f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b41512e0 0x7f61b40c4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43948/0 (socket says 172.21.15.38:43948) 2024-08-15T18:01:39.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61ba321700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b415b280 0x7f61b40c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61b9b20700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b40c8540 0x7f61b4069cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:59106/0 (socket says 172.21.15.38:59106) 2024-08-15T18:01:39.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61b931f700 1 -- 172.21.15.38:0/1299928714 learned_addr learned my addr 172.21.15.38:0/1299928714 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:39.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61b931f700 1 -- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b40c8540 msgr2=0x7f61b4069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61b931f700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b40c8540 0x7f61b4069cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61b931f700 1 -- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b415b280 msgr2=0x7f61b40c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61b931f700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b415b280 0x7f61b40c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61b931f700 1 -- 172.21.15.38:0/1299928714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b0009ae0 con 0x7f61b41512e0 2024-08-15T18:01:39.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61b931f700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b41512e0 0x7f61b40c4fb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f61a400c810 tx=0x7f61a4007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61a9ffb700 1 -- 172.21.15.38:0/1299928714 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4011590 con 0x7f61b41512e0 2024-08-15T18:01:39.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61a9ffb700 1 -- 172.21.15.38:0/1299928714 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a4011710 con 0x7f61b41512e0 2024-08-15T18:01:39.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61bab22700 1 -- 172.21.15.38:0/1299928714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b40694c0 con 0x7f61b41512e0 2024-08-15T18:01:39.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61a9ffb700 1 -- 172.21.15.38:0/1299928714 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4011ad0 con 0x7f61b41512e0 2024-08-15T18:01:39.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61bab22700 1 -- 172.21.15.38:0/1299928714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b40c7d30 con 0x7f61b41512e0 2024-08-15T18:01:39.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.002+0000 7f61bab22700 1 -- 172.21.15.38:0/1299928714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b40638f0 con 0x7f61b41512e0 2024-08-15T18:01:39.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.006+0000 7f61a9ffb700 1 -- 172.21.15.38:0/1299928714 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f61a401b3b0 con 0x7f61b41512e0 2024-08-15T18:01:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.006+0000 7f61a9ffb700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6198060860 0x7f6198062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.006+0000 7f61a9ffb700 1 -- 172.21.15.38:0/1299928714 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f61a407c4b0 con 0x7f61b41512e0 2024-08-15T18:01:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.006+0000 7f61b9b20700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6198060860 0x7f6198062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.006+0000 7f61b9b20700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6198060860 0x7f6198062d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f61b000c010 tx=0x7f61b0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.010+0000 7f61a9ffb700 1 -- 172.21.15.38:0/1299928714 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f61a404c4e0 con 0x7f61b41512e0 2024-08-15T18:01:39.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.170+0000 7f61bab22700 1 -- 172.21.15.38:0/1299928714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f61b40c9800 con 0x7f61b41512e0 2024-08-15T18:01:39.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.170+0000 7f61a9ffb700 1 -- 172.21.15.38:0/1299928714 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f61a4017020 con 0x7f61b41512e0 2024-08-15T18:01:39.173 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-08-15T18:01:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 -- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6198060860 msgr2=0x7f6198062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6198060860 0x7f6198062d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f61b000c010 tx=0x7f61b0017000 comp rx=0 tx=0).stop 2024-08-15T18:01:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 -- 172.21.15.38:0/1299928714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b41512e0 msgr2=0x7f61b40c4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b41512e0 0x7f61b40c4fb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f61a400c810 tx=0x7f61a4007ee0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 -- 172.21.15.38:0/1299928714 shutdown_connections 2024-08-15T18:01:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6198060860 0x7f6198062d10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b415b280 0x7f61b40c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b41512e0 0x7f61b40c4fb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 --2- 172.21.15.38:0/1299928714 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b40c8540 0x7f61b4069cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 -- 172.21.15.38:0/1299928714 >> 172.21.15.38:0/1299928714 conn(0x7f61b40cdf00 msgr2=0x7f61b40b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 -- 172.21.15.38:0/1299928714 shutdown_connections 2024-08-15T18:01:39.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.174+0000 7f61bab22700 1 -- 172.21.15.38:0/1299928714 wait complete. 2024-08-15T18:01:39.187 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000845064.meta pg_num 2024-08-15T18:01:39.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.302+0000 7f4217fff700 1 Processor -- start 2024-08-15T18:01:39.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.302+0000 7f4217fff700 1 -- start start 2024-08-15T18:01:39.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f4217fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f42181501b0 0x7f4218150580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f4217fff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421815a3b0 con 0x7f42180ab860 2024-08-15T18:01:39.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f4217fff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4218159ac0 con 0x7f42180686a0 2024-08-15T18:01:39.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f4217fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4218159d60 con 0x7f42181501b0 2024-08-15T18:01:39.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f420ffff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f42180ab860 0x7f4218156e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50492/0 (socket says 172.21.15.38:50492) 2024-08-15T18:01:39.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f420ffff700 1 -- 172.21.15.38:0/559612911 learned_addr learned my addr 172.21.15.38:0/559612911 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:39.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f4216ffd700 1 --2- 172.21.15.38:0/559612911 >> v2:172.21.15.104:3300/0 conn(0x7f42181501b0 0x7f4218150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f42167fc700 1 -- 172.21.15.38:0/559612911 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987526372 0 0) 0x7f421815a3b0 con 0x7f42180ab860 2024-08-15T18:01:39.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f42167fc700 1 -- 172.21.15.38:0/559612911 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f8003570 con 0x7f42180ab860 2024-08-15T18:01:39.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f4216ffd700 1 -- 172.21.15.38:0/559612911 >> v1:172.21.15.104:6789/0 conn(0x7f42180686a0 legacy=0x7f42180af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f4216ffd700 1 -- 172.21.15.38:0/559612911 >> v1:172.21.15.38:6789/0 conn(0x7f42180ab860 legacy=0x7f4218156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f4216ffd700 1 -- 172.21.15.38:0/559612911 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421815a040 con 0x7f42181501b0 2024-08-15T18:01:39.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f4216ffd700 1 --2- 172.21.15.38:0/559612911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42181501b0 0x7f4218150580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f42080098e0 tx=0x7f42080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aad1f597515fff0e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f42167fc700 1 -- 172.21.15.38:0/559612911 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420800b640 con 0x7f42181501b0 2024-08-15T18:01:39.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f42167fc700 1 -- 172.21.15.38:0/559612911 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420800b7a0 con 0x7f42181501b0 2024-08-15T18:01:39.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.306+0000 7f42167fc700 1 -- 172.21.15.38:0/559612911 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420800bb20 con 0x7f42181501b0 2024-08-15T18:01:39.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.310+0000 7f4217fff700 1 -- 172.21.15.38:0/559612911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42181501b0 msgr2=0x7f4218150580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.310+0000 7f4217fff700 1 --2- 172.21.15.38:0/559612911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42181501b0 0x7f4218150580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f42080098e0 tx=0x7f42080091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.310+0000 7f4217fff700 1 -- 172.21.15.38:0/559612911 shutdown_connections 2024-08-15T18:01:39.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.310+0000 7f4217fff700 1 --2- 172.21.15.38:0/559612911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42181501b0 0x7f4218150580 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.310+0000 7f4217fff700 1 -- 172.21.15.38:0/559612911 >> 172.21.15.38:0/559612911 conn(0x7f421806e350 msgr2=0x7f421806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:39.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.310+0000 7f4217fff700 1 -- 172.21.15.38:0/559612911 shutdown_connections 2024-08-15T18:01:39.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.310+0000 7f4217fff700 1 -- 172.21.15.38:0/559612911 wait complete. 2024-08-15T18:01:39.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.310+0000 7f4217fff700 1 Processor -- start 2024-08-15T18:01:39.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.310+0000 7f4217fff700 1 -- start start 2024-08-15T18:01:39.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4217fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42180686a0 0x7f4218145070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4217fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42180ab860 0x7f42180b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4217fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42181501b0 0x7f42180b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4217fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421814ec10 con 0x7f42180686a0 2024-08-15T18:01:39.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4217fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42181512c0 con 0x7f42180ab860 2024-08-15T18:01:39.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4217fff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42181455b0 con 0x7f42181501b0 2024-08-15T18:01:39.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4216ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42180686a0 0x7f4218145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4216ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42180686a0 0x7f4218145070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43960/0 (socket says 172.21.15.38:43960) 2024-08-15T18:01:39.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4216ffd700 1 -- 172.21.15.38:0/2357478064 learned_addr learned my addr 172.21.15.38:0/2357478064 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:39.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f420ffff700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42180ab860 0x7f42180b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f42177fe700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42181501b0 0x7f42180b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4216ffd700 1 -- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42181501b0 msgr2=0x7f42180b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4216ffd700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42181501b0 0x7f42180b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4216ffd700 1 -- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42180ab860 msgr2=0x7f42180b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4216ffd700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42180ab860 0x7f42180b03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4216ffd700 1 -- 172.21.15.38:0/2357478064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4208009420 con 0x7f42180686a0 2024-08-15T18:01:39.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4216ffd700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42180686a0 0x7f4218145070 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f42080093f0 tx=0x7f4208017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f420effd700 1 -- 172.21.15.38:0/2357478064 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208000d10 con 0x7f42180686a0 2024-08-15T18:01:39.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f420effd700 1 -- 172.21.15.38:0/2357478064 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420800b910 con 0x7f42180686a0 2024-08-15T18:01:39.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f420effd700 1 -- 172.21.15.38:0/2357478064 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208025650 con 0x7f42180686a0 2024-08-15T18:01:39.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4217fff700 1 -- 172.21.15.38:0/2357478064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421806ae70 con 0x7f42180686a0 2024-08-15T18:01:39.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.314+0000 7f4217fff700 1 -- 172.21.15.38:0/2357478064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421806b430 con 0x7f42180686a0 2024-08-15T18:01:39.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.318+0000 7f4217fff700 1 -- 172.21.15.38:0/2357478064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42180638f0 con 0x7f42180686a0 2024-08-15T18:01:39.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.318+0000 7f420effd700 1 -- 172.21.15.38:0/2357478064 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4208023070 con 0x7f42180686a0 2024-08-15T18:01:39.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.318+0000 7f420effd700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f41fc060860 0x7f41fc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.318+0000 7f420effd700 1 -- 172.21.15.38:0/2357478064 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f420807e3c0 con 0x7f42180686a0 2024-08-15T18:01:39.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.318+0000 7f420ffff700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f41fc060860 0x7f41fc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.318+0000 7f420ffff700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f41fc060860 0x7f41fc062d10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4218168590 tx=0x7f4200009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.326+0000 7f420effd700 1 -- 172.21.15.38:0/2357478064 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f420804e3f0 con 0x7f42180686a0 2024-08-15T18:01:39.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.482+0000 7f4217fff700 1 -- 172.21.15.38:0/2357478064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000845064.meta", "var": "pg_num"} v 0) v1 -- 0x7f4218151090 con 0x7f42180686a0 2024-08-15T18:01:39.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f420effd700 1 -- 172.21.15.38:0/2357478064 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000845064.meta", "var": "pg_num"}]=0 v37) v1 ==== 124+0+11 (secure 0 0 0) 0x7f4208051a10 con 0x7f42180686a0 2024-08-15T18:01:39.487 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 -- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f41fc060860 msgr2=0x7f41fc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f41fc060860 0x7f41fc062d10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4218168590 tx=0x7f4200009040 comp rx=0 tx=0).stop 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 -- 172.21.15.38:0/2357478064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42180686a0 msgr2=0x7f4218145070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42180686a0 0x7f4218145070 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f42080093f0 tx=0x7f4208017000 comp rx=0 tx=0).stop 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 -- 172.21.15.38:0/2357478064 shutdown_connections 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f41fc060860 0x7f41fc062d10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42181501b0 0x7f42180b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42180ab860 0x7f42180b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 --2- 172.21.15.38:0/2357478064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42180686a0 0x7f4218145070 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 -- 172.21.15.38:0/2357478064 >> 172.21.15.38:0/2357478064 conn(0x7f421806e350 msgr2=0x7f42180af540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 -- 172.21.15.38:0/2357478064 shutdown_connections 2024-08-15T18:01:39.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.486+0000 7f4217fff700 1 -- 172.21.15.38:0/2357478064 wait complete. 2024-08-15T18:01:39.502 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000845064.data pg_num 2024-08-15T18:01:39.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.618+0000 7f205ed90700 1 Processor -- start 2024-08-15T18:01:39.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.618+0000 7f205ed90700 1 -- start start 2024-08-15T18:01:39.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.618+0000 7f205ed90700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2058068db0 0x7f2058155820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.618+0000 7f205ed90700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20580afe30 con 0x7f205806e870 2024-08-15T18:01:39.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.618+0000 7f205ed90700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20580ac330 con 0x7f2058155450 2024-08-15T18:01:39.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.618+0000 7f205ed90700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20580ac590 con 0x7f2058068db0 2024-08-15T18:01:39.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205d58d700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f205806e870 0x7f20580686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50500/0 (socket says 172.21.15.38:50500) 2024-08-15T18:01:39.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205d58d700 1 -- 172.21.15.38:0/4225714917 learned_addr learned my addr 172.21.15.38:0/4225714917 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:39.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205e58f700 1 --2- 172.21.15.38:0/4225714917 >> v2:172.21.15.104:3300/0 conn(0x7f2058068db0 0x7f2058155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205e58f700 1 -- 172.21.15.38:0/4225714917 >> v1:172.21.15.104:6789/0 conn(0x7f2058155450 legacy=0x7f20580abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205cd8c700 1 -- 172.21.15.38:0/4225714917 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233153671 0 0) 0x7f20580afe30 con 0x7f205806e870 2024-08-15T18:01:39.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205e58f700 1 -- 172.21.15.38:0/4225714917 >> v1:172.21.15.38:6789/0 conn(0x7f205806e870 legacy=0x7f20580686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205e58f700 1 -- 172.21.15.38:0/4225714917 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20580af500 con 0x7f2058068db0 2024-08-15T18:01:39.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205e58f700 1 --2- 172.21.15.38:0/4225714917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2058068db0 0x7f2058155820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2050009e60 tx=0x7f2050009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e36178249acf0a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205cd8c700 1 -- 172.21.15.38:0/4225714917 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205000bc30 con 0x7f2058068db0 2024-08-15T18:01:39.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205cd8c700 1 -- 172.21.15.38:0/4225714917 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205000bd90 con 0x7f2058068db0 2024-08-15T18:01:39.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205cd8c700 1 -- 172.21.15.38:0/4225714917 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20500116b0 con 0x7f2058068db0 2024-08-15T18:01:39.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205ed90700 1 -- 172.21.15.38:0/4225714917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2058068db0 msgr2=0x7f2058155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205ed90700 1 --2- 172.21.15.38:0/4225714917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2058068db0 0x7f2058155820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2050009e60 tx=0x7f2050009780 comp rx=0 tx=0).stop 2024-08-15T18:01:39.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205ed90700 1 -- 172.21.15.38:0/4225714917 shutdown_connections 2024-08-15T18:01:39.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205ed90700 1 --2- 172.21.15.38:0/4225714917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2058068db0 0x7f2058155820 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205ed90700 1 -- 172.21.15.38:0/4225714917 >> 172.21.15.38:0/4225714917 conn(0x7f20580ccfc0 msgr2=0x7f205806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:39.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205ed90700 1 -- 172.21.15.38:0/4225714917 shutdown_connections 2024-08-15T18:01:39.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.622+0000 7f205ed90700 1 -- 172.21.15.38:0/4225714917 wait complete. 2024-08-15T18:01:39.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205ed90700 1 Processor -- start 2024-08-15T18:01:39.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205ed90700 1 -- start start 2024-08-15T18:01:39.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205ed90700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2058068db0 0x7f20581398d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205ed90700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f205806e870 0x7f2058153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205ed90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2058155450 0x7f2058154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205ed90700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2058154590 con 0x7f2058155450 2024-08-15T18:01:39.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205ed90700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2058151b50 con 0x7f2058068db0 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205ed90700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2058139e10 con 0x7f205806e870 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205e58f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2058155450 0x7f2058154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205e58f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2058155450 0x7f2058154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43970/0 (socket says 172.21.15.38:43970) 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205e58f700 1 -- 172.21.15.38:0/40398136 learned_addr learned my addr 172.21.15.38:0/40398136 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205d58d700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f205806e870 0x7f2058153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205dd8e700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2058068db0 0x7f20581398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205e58f700 1 -- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f205806e870 msgr2=0x7f2058153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205e58f700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f205806e870 0x7f2058153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205e58f700 1 -- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2058068db0 msgr2=0x7f20581398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205e58f700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2058068db0 0x7f20581398d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205e58f700 1 -- 172.21.15.38:0/40398136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2050009a20 con 0x7f2058155450 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205e58f700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2058155450 0x7f2058154050 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f20580ac4c0 tx=0x7f205001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f204dffb700 1 -- 172.21.15.38:0/40398136 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20500032f0 con 0x7f2058155450 2024-08-15T18:01:39.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f204dffb700 1 -- 172.21.15.38:0/40398136 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205001e050 con 0x7f2058155450 2024-08-15T18:01:39.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.626+0000 7f205ed90700 1 -- 172.21.15.38:0/40398136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205814fe10 con 0x7f2058155450 2024-08-15T18:01:39.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.630+0000 7f204dffb700 1 -- 172.21.15.38:0/40398136 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205001ae80 con 0x7f2058155450 2024-08-15T18:01:39.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.630+0000 7f205ed90700 1 -- 172.21.15.38:0/40398136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20581517e0 con 0x7f2058155450 2024-08-15T18:01:39.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.630+0000 7f205ed90700 1 -- 172.21.15.38:0/40398136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2058151ce0 con 0x7f2058155450 2024-08-15T18:01:39.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.630+0000 7f204dffb700 1 -- 172.21.15.38:0/40398136 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2050003040 con 0x7f2058155450 2024-08-15T18:01:39.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.630+0000 7f204dffb700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f203c060860 0x7f203c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.630+0000 7f204dffb700 1 -- 172.21.15.38:0/40398136 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f2050081e00 con 0x7f2058155450 2024-08-15T18:01:39.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.630+0000 7f205dd8e700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f203c060860 0x7f203c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.634+0000 7f205dd8e700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f203c060860 0x7f203c062d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2054000c00 tx=0x7f2054009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.638+0000 7f204dffb700 1 -- 172.21.15.38:0/40398136 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2050051ed0 con 0x7f2058155450 2024-08-15T18:01:39.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 -- 172.21.15.38:0/40398136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000845064.data", "var": "pg_num"} v 0) v1 -- 0x7f20580638f0 con 0x7f2058155450 2024-08-15T18:01:39.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f204dffb700 1 -- 172.21.15.38:0/40398136 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000845064.data", "var": "pg_num"}]=0 v37) v1 ==== 124+0+11 (secure 0 0 0) 0x7f20500554f0 con 0x7f2058155450 2024-08-15T18:01:39.804 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T18:01:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 -- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f203c060860 msgr2=0x7f203c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f203c060860 0x7f203c062d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2054000c00 tx=0x7f2054009040 comp rx=0 tx=0).stop 2024-08-15T18:01:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 -- 172.21.15.38:0/40398136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2058155450 msgr2=0x7f2058154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2058155450 0x7f2058154050 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f20580ac4c0 tx=0x7f205001b000 comp rx=0 tx=0).stop 2024-08-15T18:01:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 -- 172.21.15.38:0/40398136 shutdown_connections 2024-08-15T18:01:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f203c060860 0x7f203c062d10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2058155450 0x7f2058154050 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f205806e870 0x7f2058153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 --2- 172.21.15.38:0/40398136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2058068db0 0x7f20581398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 -- 172.21.15.38:0/40398136 >> 172.21.15.38:0/40398136 conn(0x7f20580ccfc0 msgr2=0x7f20580b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 -- 172.21.15.38:0/40398136 shutdown_connections 2024-08-15T18:01:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.802+0000 7f205ed90700 1 -- 172.21.15.38:0/40398136 wait complete. 2024-08-15T18:01:39.819 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T18:01:39.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.918+0000 7fe025458700 1 Processor -- start 2024-08-15T18:01:39.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.918+0000 7fe025458700 1 -- start start 2024-08-15T18:01:39.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe020142890 0x7fe0200cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe02015d5b0 0x7fe0200cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0200cc800 con 0x7fe020142c60 2024-08-15T18:01:39.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200b1610 con 0x7fe020142890 2024-08-15T18:01:39.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200cbbf0 con 0x7fe02015d5b0 2024-08-15T18:01:39.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01effd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe020142890 0x7fe0200cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01e7fc700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe020142c60 0x7fe02015cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50502/0 (socket says 172.21.15.38:50502) 2024-08-15T18:01:39.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01e7fc700 1 -- 172.21.15.38:0/3140985648 learned_addr learned my addr 172.21.15.38:0/3140985648 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:39.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01f7fe700 1 --2- 172.21.15.38:0/3140985648 >> v2:172.21.15.104:3300/0 conn(0x7fe02015d5b0 0x7fe0200cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01effd700 1 -- 172.21.15.38:0/3140985648 >> v2:172.21.15.104:3300/0 conn(0x7fe02015d5b0 msgr2=0x7fe0200cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:39.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01effd700 1 --2- 172.21.15.38:0/3140985648 >> v2:172.21.15.104:3300/0 conn(0x7fe02015d5b0 0x7fe0200cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01effd700 1 -- 172.21.15.38:0/3140985648 >> v1:172.21.15.38:6789/0 conn(0x7fe020142c60 legacy=0x7fe02015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01effd700 1 -- 172.21.15.38:0/3140985648 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0200cbed0 con 0x7fe020142890 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01effd700 1 --2- 172.21.15.38:0/3140985648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe020142890 0x7fe0200cb0a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe0100098e0 tx=0x7fe0100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82039fc2efcf06a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01dffb700 1 -- 172.21.15.38:0/3140985648 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010011630 con 0x7fe020142890 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01dffb700 1 -- 172.21.15.38:0/3140985648 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010011790 con 0x7fe020142890 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe01dffb700 1 -- 172.21.15.38:0/3140985648 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010011b30 con 0x7fe020142890 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 -- 172.21.15.38:0/3140985648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe020142890 msgr2=0x7fe0200cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 --2- 172.21.15.38:0/3140985648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe020142890 0x7fe0200cb0a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe0100098e0 tx=0x7fe0100091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 -- 172.21.15.38:0/3140985648 shutdown_connections 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 --2- 172.21.15.38:0/3140985648 >> v2:172.21.15.104:3300/0 conn(0x7fe02015d5b0 0x7fe0200cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 --2- 172.21.15.38:0/3140985648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe020142890 0x7fe0200cb0a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 -- 172.21.15.38:0/3140985648 >> 172.21.15.38:0/3140985648 conn(0x7fe0200a9db0 msgr2=0x7fe0200ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:39.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 -- 172.21.15.38:0/3140985648 shutdown_connections 2024-08-15T18:01:39.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.922+0000 7fe025458700 1 -- 172.21.15.38:0/3140985648 wait complete. 2024-08-15T18:01:39.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.926+0000 7fe025458700 1 Processor -- start 2024-08-15T18:01:39.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.926+0000 7fe025458700 1 -- start start 2024-08-15T18:01:39.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe025458700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe020142890 0x7fe020068640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe025458700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe020142c60 0x7fe02016a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe025458700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe02015d5b0 0x7fe0200c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe025458700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02016a5d0 con 0x7fe020142890 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe025458700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02016b7d0 con 0x7fe02015d5b0 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe025458700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200c8c40 con 0x7fe020142c60 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe020142890 0x7fe020068640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe020142890 0x7fe020068640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43988/0 (socket says 172.21.15.38:43988) 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01effd700 1 -- 172.21.15.38:0/119839423 learned_addr learned my addr 172.21.15.38:0/119839423 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01e7fc700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe020142c60 0x7fe02016a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01f7fe700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe02015d5b0 0x7fe0200c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01effd700 1 -- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe020142c60 msgr2=0x7fe02016a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01effd700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe020142c60 0x7fe02016a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01effd700 1 -- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe02015d5b0 msgr2=0x7fe0200c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01effd700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe02015d5b0 0x7fe0200c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:39.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01effd700 1 -- 172.21.15.38:0/119839423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe010009420 con 0x7fe020142890 2024-08-15T18:01:39.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01e7fc700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe020142c60 0x7fe02016a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:39.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01f7fe700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe02015d5b0 0x7fe0200c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe01effd700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe020142890 0x7fe020068640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe0100091e0 tx=0x7fe010017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe006ffd700 1 -- 172.21.15.38:0/119839423 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01001bc10 con 0x7fe020142890 2024-08-15T18:01:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe006ffd700 1 -- 172.21.15.38:0/119839423 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01001bd90 con 0x7fe020142890 2024-08-15T18:01:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe025458700 1 -- 172.21.15.38:0/119839423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02016abc0 con 0x7fe020142890 2024-08-15T18:01:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe006ffd700 1 -- 172.21.15.38:0/119839423 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010025670 con 0x7fe020142890 2024-08-15T18:01:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.930+0000 7fe025458700 1 -- 172.21.15.38:0/119839423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02016b180 con 0x7fe020142890 2024-08-15T18:01:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.934+0000 7fe025458700 1 -- 172.21.15.38:0/119839423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0200645c0 con 0x7fe020142890 2024-08-15T18:01:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.934+0000 7fe006ffd700 1 -- 172.21.15.38:0/119839423 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe010023070 con 0x7fe020142890 2024-08-15T18:01:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.934+0000 7fe006ffd700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe00c060860 0x7fe00c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:39.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.934+0000 7fe006ffd700 1 -- 172.21.15.38:0/119839423 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fe01007e440 con 0x7fe020142890 2024-08-15T18:01:39.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.934+0000 7fe01e7fc700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe00c060860 0x7fe00c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:39.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.934+0000 7fe01e7fc700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe00c060860 0x7fe00c062d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe02013f8c0 tx=0x7fe014009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:39.942+0000 7fe006ffd700 1 -- 172.21.15.38:0/119839423 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe01004e5b0 con 0x7fe020142890 2024-08-15T18:01:40.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.142+0000 7fe025458700 1 -- 172.21.15.38:0/119839423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0200aff80 con 0x7fe020142890 2024-08-15T18:01:40.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.142+0000 7fe006ffd700 1 -- 172.21.15.38:0/119839423 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5681 (secure 0 0 0) 0x7fe010051b90 con 0x7fe020142890 2024-08-15T18:01:40.144 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:40.145 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/373633111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":373633111},{"type":"v1","addr":"172.21.15.38:6833","nonce":373633111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/389886189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":389886189},{"type":"v1","addr":"172.21.15.38:6835","nonce":389886189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:20.353438+0000","modified":"2024-08-15T17:59:24.015140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4999},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4999":{"gid":4999,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/1865762791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1865762791},{"type":"v1","addr":"172.21.15.104:6837","nonce":1865762791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:35.121185+0000","modified":"2024-08-15T17:59:36.892161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4981":{"gid":4981,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6835/875828080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":875828080},{"type":"v1","addr":"172.21.15.104:6835","nonce":875828080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000845064","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T18:01:40.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.142+0000 7fe025458700 1 -- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe00c060860 msgr2=0x7fe00c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.142+0000 7fe025458700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe00c060860 0x7fe00c062d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe02013f8c0 tx=0x7fe014009040 comp rx=0 tx=0).stop 2024-08-15T18:01:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.142+0000 7fe025458700 1 -- 172.21.15.38:0/119839423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe020142890 msgr2=0x7fe020068640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.142+0000 7fe025458700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe020142890 0x7fe020068640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe0100091e0 tx=0x7fe010017000 comp rx=0 tx=0).stop 2024-08-15T18:01:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.142+0000 7fe025458700 1 -- 172.21.15.38:0/119839423 shutdown_connections 2024-08-15T18:01:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.142+0000 7fe025458700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe00c060860 0x7fe00c062d10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.142+0000 7fe025458700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe02015d5b0 0x7fe0200c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.142+0000 7fe025458700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe020142c60 0x7fe02016a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.142+0000 7fe025458700 1 --2- 172.21.15.38:0/119839423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe020142890 0x7fe020068640 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.146+0000 7fe025458700 1 -- 172.21.15.38:0/119839423 >> 172.21.15.38:0/119839423 conn(0x7fe0200a9db0 msgr2=0x7fe0200ac190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.146+0000 7fe025458700 1 -- 172.21.15.38:0/119839423 shutdown_connections 2024-08-15T18:01:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.146+0000 7fe025458700 1 -- 172.21.15.38:0/119839423 wait complete. 2024-08-15T18:01:40.149 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 18 2024-08-15T18:01:40.159 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T18:01:40.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f81f18700 1 Processor -- start 2024-08-15T18:01:40.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f81f18700 1 -- start start 2024-08-15T18:01:40.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f81f18700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2f7c155e50 0x7f2f7c0a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f81f18700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f7c0afc10 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f81f18700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f7c0aed20 con 0x7f2f7c156220 2024-08-15T18:01:40.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f81f18700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7c0af000 con 0x7f2f7c155e50 2024-08-15T18:01:40.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7bfff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2f7c06f2f0 0x7f2f7c0b00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50516/0 (socket says 172.21.15.38:50516) 2024-08-15T18:01:40.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7bfff700 1 -- 172.21.15.38:0/1985866662 learned_addr learned my addr 172.21.15.38:0/1985866662 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:40.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7b7fe700 1 --2- 172.21.15.38:0/1985866662 >> v2:172.21.15.104:3300/0 conn(0x7f2f7c155e50 0x7f2f7c0a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7a7fc700 1 -- 172.21.15.38:0/1985866662 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082595995 0 0) 0x7f2f7c0afc10 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7a7fc700 1 -- 172.21.15.38:0/1985866662 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f68003570 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7a7fc700 1 -- 172.21.15.38:0/1985866662 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369907118 0 0) 0x7f2f7c0aed20 con 0x7f2f7c156220 2024-08-15T18:01:40.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7a7fc700 1 -- 172.21.15.38:0/1985866662 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f7c0afc10 con 0x7f2f7c156220 2024-08-15T18:01:40.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7b7fe700 1 -- 172.21.15.38:0/1985866662 >> v1:172.21.15.104:6789/0 conn(0x7f2f7c156220 legacy=0x7f2f7c06ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7b7fe700 1 -- 172.21.15.38:0/1985866662 >> v1:172.21.15.38:6789/0 conn(0x7f2f7c06f2f0 legacy=0x7f2f7c0b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7a7fc700 1 -- 172.21.15.38:0/1985866662 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 929369946 0 0) 0x7f2f68003570 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7b7fe700 1 -- 172.21.15.38:0/1985866662 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c0af2e0 con 0x7f2f7c155e50 2024-08-15T18:01:40.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7b7fe700 1 --2- 172.21.15.38:0/1985866662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f7c155e50 0x7f2f7c0a9ee0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2f640098e0 tx=0x7f2f640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33115f3ea6c33b3f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:40.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.262+0000 7f2f7a7fc700 1 -- 172.21.15.38:0/1985866662 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740017f0 con 0x7f2f7c155e50 2024-08-15T18:01:40.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.266+0000 7f2f7a7fc700 1 -- 172.21.15.38:0/1985866662 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f64011420 con 0x7f2f7c155e50 2024-08-15T18:01:40.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.266+0000 7f2f7a7fc700 1 -- 172.21.15.38:0/1985866662 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f640117c0 con 0x7f2f7c155e50 2024-08-15T18:01:40.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.266+0000 7f2f81f18700 1 -- 172.21.15.38:0/1985866662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f7c155e50 msgr2=0x7f2f7c0a9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.266+0000 7f2f81f18700 1 --2- 172.21.15.38:0/1985866662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f7c155e50 0x7f2f7c0a9ee0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2f640098e0 tx=0x7f2f640091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.266+0000 7f2f81f18700 1 -- 172.21.15.38:0/1985866662 shutdown_connections 2024-08-15T18:01:40.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.266+0000 7f2f81f18700 1 --2- 172.21.15.38:0/1985866662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f7c155e50 0x7f2f7c0a9ee0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.266+0000 7f2f81f18700 1 -- 172.21.15.38:0/1985866662 >> 172.21.15.38:0/1985866662 conn(0x7f2f7c066e00 msgr2=0x7f2f7c067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:40.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.266+0000 7f2f81f18700 1 -- 172.21.15.38:0/1985866662 shutdown_connections 2024-08-15T18:01:40.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.266+0000 7f2f81f18700 1 -- 172.21.15.38:0/1985866662 wait complete. 2024-08-15T18:01:40.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.266+0000 7f2f81f18700 1 Processor -- start 2024-08-15T18:01:40.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.266+0000 7f2f81f18700 1 -- start start 2024-08-15T18:01:40.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f81f18700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f7c06f2f0 0x7f2f7c14df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f81f18700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f7c155e50 0x7f2f7c1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f81f18700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f7c156220 0x7f2f7c150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f81f18700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c14e4a0 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f81f18700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c166500 con 0x7f2f7c156220 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f81f18700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c165bd0 con 0x7f2f7c155e50 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f7c06f2f0 0x7f2f7c14df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f7c06f2f0 0x7f2f7c14df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44002/0 (socket says 172.21.15.38:44002) 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7b7fe700 1 -- 172.21.15.38:0/951879043 learned_addr learned my addr 172.21.15.38:0/951879043 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7affd700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f7c155e50 0x7f2f7c1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7bfff700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f7c156220 0x7f2f7c150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7b7fe700 1 -- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f7c155e50 msgr2=0x7f2f7c1501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7b7fe700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f7c155e50 0x7f2f7c1501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7b7fe700 1 -- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f7c156220 msgr2=0x7f2f7c150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7b7fe700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f7c156220 0x7f2f7c150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7b7fe700 1 -- 172.21.15.38:0/951879043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64009420 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7affd700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f7c155e50 0x7f2f7c1501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f7b7fe700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f7c06f2f0 0x7f2f7c14df60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c06f870 tx=0x7f2f64016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f737fe700 1 -- 172.21.15.38:0/951879043 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6401cc90 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f737fe700 1 -- 172.21.15.38:0/951879043 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6401cdf0 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f81f18700 1 -- 172.21.15.38:0/951879043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c165eb0 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f737fe700 1 -- 172.21.15.38:0/951879043 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64025680 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f81f18700 1 -- 172.21.15.38:0/951879043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c165840 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f81f18700 1 -- 172.21.15.38:0/951879043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7c0645c0 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f737fe700 1 -- 172.21.15.38:0/951879043 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2f640193f0 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.270+0000 7f2f737fe700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f5c060860 0x7f2f5c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.274+0000 7f2f737fe700 1 -- 172.21.15.38:0/951879043 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f2f6407e2e0 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.274+0000 7f2f7affd700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f5c060860 0x7f2f5c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.274+0000 7f2f7affd700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f5c060860 0x7f2f5c062d10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0c7e50 tx=0x7f2f6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:40.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.278+0000 7f2f737fe700 1 -- 172.21.15.38:0/951879043 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2f6404e430 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.446+0000 7f2f81f18700 1 -- 172.21.15.38:0/951879043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f7c099a20 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.446+0000 7f2f737fe700 1 -- 172.21.15.38:0/951879043 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18599 (secure 0 0 0) 0x7f2f64051a50 con 0x7f2f7c06f2f0 2024-08-15T18:01:40.451 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:40.451 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":37,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:59:38.114973+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:59:18.793510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:59:19.328139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000845064.meta","create_time":"2024-08-15T17:59:33.132275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000845064"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000845064.data","create_time":"2024-08-15T17:59:34.143101+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000845064"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T18:01:40.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 -- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f5c060860 msgr2=0x7f2f5c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f5c060860 0x7f2f5c062d10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0c7e50 tx=0x7f2f6c009040 comp rx=0 tx=0).stop 2024-08-15T18:01:40.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 -- 172.21.15.38:0/951879043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f7c06f2f0 msgr2=0x7f2f7c14df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f7c06f2f0 0x7f2f7c14df60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c06f870 tx=0x7f2f64016000 comp rx=0 tx=0).stop 2024-08-15T18:01:40.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 -- 172.21.15.38:0/951879043 shutdown_connections 2024-08-15T18:01:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2f5c060860 0x7f2f5c062d10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f7c156220 0x7f2f7c150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f7c155e50 0x7f2f7c1501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 --2- 172.21.15.38:0/951879043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f7c06f2f0 0x7f2f7c14df60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 -- 172.21.15.38:0/951879043 >> 172.21.15.38:0/951879043 conn(0x7f2f7c066e00 msgr2=0x7f2f7c153ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 -- 172.21.15.38:0/951879043 shutdown_connections 2024-08-15T18:01:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.450+0000 7f2f81f18700 1 -- 172.21.15.38:0/951879043 wait complete. 2024-08-15T18:01:40.465 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-15T18:01:40.465 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T18:01:40.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.570+0000 7f5efa369700 1 Processor -- start 2024-08-15T18:01:40.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.570+0000 7f5efa369700 1 -- start start 2024-08-15T18:01:40.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.570+0000 7f5efa369700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5ef40d0010 0x7f5ef40d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.570+0000 7f5efa369700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ef415cea0 0x7f5ef415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.570+0000 7f5efa369700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef40cf190 con 0x7f5ef4159040 2024-08-15T18:01:40.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.570+0000 7f5efa369700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef40d09a0 con 0x7f5ef40d0010 2024-08-15T18:01:40.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.570+0000 7f5efa369700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40ce5e0 con 0x7f5ef415cea0 2024-08-15T18:01:40.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.570+0000 7f5ef9367700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5ef4159040 0x7f5ef40cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50520/0 (socket says 172.21.15.38:50520) 2024-08-15T18:01:40.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.570+0000 7f5ef9367700 1 -- 172.21.15.38:0/1332916860 learned_addr learned my addr 172.21.15.38:0/1332916860 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:40.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5ef9b68700 1 --2- 172.21.15.38:0/1332916860 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ef415cea0 0x7f5ef415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5ef8b66700 1 --2- 172.21.15.38:0/1332916860 >> v2:172.21.15.104:3300/0 conn(0x7f5ef40d0010 0x7f5ef40d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5eebfff700 1 -- 172.21.15.38:0/1332916860 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456032228 0 0) 0x7f5ef40cf190 con 0x7f5ef4159040 2024-08-15T18:01:40.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5eebfff700 1 -- 172.21.15.38:0/1332916860 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee0003570 con 0x7f5ef4159040 2024-08-15T18:01:40.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5ef9b68700 1 -- 172.21.15.38:0/1332916860 >> v2:172.21.15.104:3300/0 conn(0x7f5ef40d0010 msgr2=0x7f5ef40d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:40.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5ef9b68700 1 --2- 172.21.15.38:0/1332916860 >> v2:172.21.15.104:3300/0 conn(0x7f5ef40d0010 0x7f5ef40d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5eebfff700 1 -- 172.21.15.38:0/1332916860 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4110042667 0 0) 0x7f5ee0003570 con 0x7f5ef4159040 2024-08-15T18:01:40.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5ef9b68700 1 -- 172.21.15.38:0/1332916860 >> v1:172.21.15.38:6789/0 conn(0x7f5ef4159040 legacy=0x7f5ef40cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5ef9b68700 1 -- 172.21.15.38:0/1332916860 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40ce860 con 0x7f5ef415cea0 2024-08-15T18:01:40.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5ef9b68700 1 --2- 172.21.15.38:0/1332916860 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ef415cea0 0x7f5ef415d320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009e60 tx=0x7f5eec009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd2cbcee93d48cf server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:40.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5eebfff700 1 -- 172.21.15.38:0/1332916860 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00e070 con 0x7f5ef415cea0 2024-08-15T18:01:40.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5eebfff700 1 -- 172.21.15.38:0/1332916860 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec00b7a0 con 0x7f5ef415cea0 2024-08-15T18:01:40.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5eebfff700 1 -- 172.21.15.38:0/1332916860 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0001470 con 0x7f5ef415cea0 2024-08-15T18:01:40.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5efa369700 1 -- 172.21.15.38:0/1332916860 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ef415cea0 msgr2=0x7f5ef415d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5efa369700 1 --2- 172.21.15.38:0/1332916860 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ef415cea0 0x7f5ef415d320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009e60 tx=0x7f5eec009780 comp rx=0 tx=0).stop 2024-08-15T18:01:40.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5efa369700 1 -- 172.21.15.38:0/1332916860 shutdown_connections 2024-08-15T18:01:40.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5efa369700 1 --2- 172.21.15.38:0/1332916860 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ef415cea0 0x7f5ef415d320 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5efa369700 1 --2- 172.21.15.38:0/1332916860 >> v2:172.21.15.104:3300/0 conn(0x7f5ef40d0010 0x7f5ef40d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5efa369700 1 -- 172.21.15.38:0/1332916860 >> 172.21.15.38:0/1332916860 conn(0x7f5ef4068250 msgr2=0x7f5ef4068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:40.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5efa369700 1 -- 172.21.15.38:0/1332916860 shutdown_connections 2024-08-15T18:01:40.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.574+0000 7f5efa369700 1 -- 172.21.15.38:0/1332916860 wait complete. 2024-08-15T18:01:40.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5efa369700 1 Processor -- start 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5efa369700 1 -- start start 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5efa369700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ef40d0010 0x7f5ef413f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5efa369700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef4159040 0x7f5ef40b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5efa369700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef415cea0 0x7f5ef415bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5efa369700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef413f700 con 0x7f5ef415cea0 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5efa369700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40b0c70 con 0x7f5ef4159040 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5efa369700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40b0f10 con 0x7f5ef40d0010 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5ef9b68700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef415cea0 0x7f5ef415bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5ef9b68700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef415cea0 0x7f5ef415bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44008/0 (socket says 172.21.15.38:44008) 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5ef9b68700 1 -- 172.21.15.38:0/3610919427 learned_addr learned my addr 172.21.15.38:0/3610919427 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5ef8b66700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef4159040 0x7f5ef40b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.578+0000 7f5ef9367700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ef40d0010 0x7f5ef413f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ef9b68700 1 -- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ef40d0010 msgr2=0x7f5ef413f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ef9b68700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ef40d0010 0x7f5ef413f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ef9b68700 1 -- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef4159040 msgr2=0x7f5ef40b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ef9b68700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef4159040 0x7f5ef40b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ef9b68700 1 -- 172.21.15.38:0/3610919427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec009a20 con 0x7f5ef415cea0 2024-08-15T18:01:40.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ef9b68700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef415cea0 0x7f5ef415bf60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40ce720 tx=0x7f5eec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:40.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ee97fa700 1 -- 172.21.15.38:0/3610919427 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec0070e0 con 0x7f5ef415cea0 2024-08-15T18:01:40.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ee97fa700 1 -- 172.21.15.38:0/3610919427 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec012af0 con 0x7f5ef415cea0 2024-08-15T18:01:40.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5efa369700 1 -- 172.21.15.38:0/3610919427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef40b11f0 con 0x7f5ef415cea0 2024-08-15T18:01:40.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ee97fa700 1 -- 172.21.15.38:0/3610919427 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec025460 con 0x7f5ef415cea0 2024-08-15T18:01:40.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5efa369700 1 -- 172.21.15.38:0/3610919427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef40b17b0 con 0x7f5ef415cea0 2024-08-15T18:01:40.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5efa369700 1 -- 172.21.15.38:0/3610919427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef40690f0 con 0x7f5ef415cea0 2024-08-15T18:01:40.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ee97fa700 1 -- 172.21.15.38:0/3610919427 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5eec01c410 con 0x7f5ef415cea0 2024-08-15T18:01:40.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ee97fa700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ed8060860 0x7f5ed8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.582+0000 7f5ee97fa700 1 -- 172.21.15.38:0/3610919427 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f5eec07ecc0 con 0x7f5ef415cea0 2024-08-15T18:01:40.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.586+0000 7f5ef9367700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ed8060860 0x7f5ed8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.586+0000 7f5ef9367700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ed8060860 0x7f5ed8062d10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5ef41554a0 tx=0x7f5ef0001970 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:40.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.594+0000 7f5ee97fa700 1 -- 172.21.15.38:0/3610919427 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5ef40690f0 con 0x7f5ef415cea0 2024-08-15T18:01:40.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.794+0000 7f5efa369700 1 -- 172.21.15.38:0/3610919427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ef40690f0 con 0x7f5ef415cea0 2024-08-15T18:01:40.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5ee97fa700 1 -- 172.21.15.38:0/3610919427 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5681 (secure 0 0 0) 0x7f5ef40690f0 con 0x7f5ef415cea0 2024-08-15T18:01:40.800 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:40.800 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/373633111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":373633111},{"type":"v1","addr":"172.21.15.38:6833","nonce":373633111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/389886189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":389886189},{"type":"v1","addr":"172.21.15.38:6835","nonce":389886189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:20.353438+0000","modified":"2024-08-15T17:59:24.015140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4999},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4999":{"gid":4999,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/1865762791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1865762791},{"type":"v1","addr":"172.21.15.104:6837","nonce":1865762791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:35.121185+0000","modified":"2024-08-15T17:59:36.892161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4981":{"gid":4981,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6835/875828080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":875828080},{"type":"v1","addr":"172.21.15.104:6835","nonce":875828080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000845064","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T18:01:40.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 -- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ed8060860 msgr2=0x7f5ed8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ed8060860 0x7f5ed8062d10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5ef41554a0 tx=0x7f5ef0001970 comp rx=0 tx=0).stop 2024-08-15T18:01:40.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 -- 172.21.15.38:0/3610919427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef415cea0 msgr2=0x7f5ef415bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef415cea0 0x7f5ef415bf60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40ce720 tx=0x7f5eec018000 comp rx=0 tx=0).stop 2024-08-15T18:01:40.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 -- 172.21.15.38:0/3610919427 shutdown_connections 2024-08-15T18:01:40.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ed8060860 0x7f5ed8062d10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef415cea0 0x7f5ef415bf60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef4159040 0x7f5ef40b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 --2- 172.21.15.38:0/3610919427 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ef40d0010 0x7f5ef413f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 -- 172.21.15.38:0/3610919427 >> 172.21.15.38:0/3610919427 conn(0x7f5ef4068250 msgr2=0x7f5ef41581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:40.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 -- 172.21.15.38:0/3610919427 shutdown_connections 2024-08-15T18:01:40.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.798+0000 7f5efa369700 1 -- 172.21.15.38:0/3610919427 wait complete. 2024-08-15T18:01:40.804 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 18 2024-08-15T18:01:40.814 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T18:01:40.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 Processor -- start 2024-08-15T18:01:40.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 -- start start 2024-08-15T18:01:40.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6b2c0d0010 0x7f6b2c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6b2c15cea0 0x7f6b2c15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c0cf190 con 0x7f6b2c159040 2024-08-15T18:01:40.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c0d09a0 con 0x7f6b2c15cea0 2024-08-15T18:01:40.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c0ce5e0 con 0x7f6b2c0d0010 2024-08-15T18:01:40.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6b2c15cea0 0x7f6b2c15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6b2c15cea0 0x7f6b2c15d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44020/0 (socket says 172.21.15.38:44020) 2024-08-15T18:01:40.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2bfff700 1 -- 172.21.15.38:0/3219180837 learned_addr learned my addr 172.21.15.38:0/3219180837 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:40.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2b7fe700 1 --1- 172.21.15.38:0/3219180837 >> v1:172.21.15.38:6789/0 conn(0x7f6b2c159040 0x7f6b2c0cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50530/0 (socket says 172.21.15.38:50530) 2024-08-15T18:01:40.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b23fff700 1 --2- 172.21.15.38:0/3219180837 >> v2:172.21.15.104:3300/0 conn(0x7f6b2c0d0010 0x7f6b2c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2bfff700 1 -- 172.21.15.38:0/3219180837 >> v2:172.21.15.104:3300/0 conn(0x7f6b2c0d0010 msgr2=0x7f6b2c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:40.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2bfff700 1 --2- 172.21.15.38:0/3219180837 >> v2:172.21.15.104:3300/0 conn(0x7f6b2c0d0010 0x7f6b2c0d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2bfff700 1 -- 172.21.15.38:0/3219180837 >> v1:172.21.15.38:6789/0 conn(0x7f6b2c159040 legacy=0x7f6b2c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2bfff700 1 -- 172.21.15.38:0/3219180837 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0ce860 con 0x7f6b2c15cea0 2024-08-15T18:01:40.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b23fff700 1 --2- 172.21.15.38:0/3219180837 >> v2:172.21.15.104:3300/0 conn(0x7f6b2c0d0010 0x7f6b2c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:40.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2bfff700 1 --2- 172.21.15.38:0/3219180837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b2c15cea0 0x7f6b2c15d2e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c067ef0 tx=0x7f6b24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c542ece44708c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:40.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2affd700 1 -- 172.21.15.38:0/3219180837 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400e070 con 0x7f6b2c15cea0 2024-08-15T18:01:40.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2affd700 1 -- 172.21.15.38:0/3219180837 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2400ba50 con 0x7f6b2c15cea0 2024-08-15T18:01:40.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b2affd700 1 -- 172.21.15.38:0/3219180837 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b240092e0 con 0x7f6b2c15cea0 2024-08-15T18:01:40.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3219180837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b2c15cea0 msgr2=0x7f6b2c15d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 --2- 172.21.15.38:0/3219180837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b2c15cea0 0x7f6b2c15d2e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c067ef0 tx=0x7f6b24009840 comp rx=0 tx=0).stop 2024-08-15T18:01:40.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3219180837 shutdown_connections 2024-08-15T18:01:40.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 --2- 172.21.15.38:0/3219180837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b2c15cea0 0x7f6b2c15d2e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 --2- 172.21.15.38:0/3219180837 >> v2:172.21.15.104:3300/0 conn(0x7f6b2c0d0010 0x7f6b2c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3219180837 >> 172.21.15.38:0/3219180837 conn(0x7f6b2c068250 msgr2=0x7f6b2c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:40.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3219180837 shutdown_connections 2024-08-15T18:01:40.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.910+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3219180837 wait complete. 2024-08-15T18:01:40.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b30bc5700 1 Processor -- start 2024-08-15T18:01:40.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b30bc5700 1 -- start start 2024-08-15T18:01:40.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b30bc5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b2c0d0010 0x7f6b2c151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b30bc5700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b2c159040 0x7f6b2c0b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b30bc5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b2c15cea0 0x7f6b2c157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b30bc5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0b2860 con 0x7f6b2c15cea0 2024-08-15T18:01:40.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b30bc5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0b1be0 con 0x7f6b2c0d0010 2024-08-15T18:01:40.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b30bc5700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0b1ec0 con 0x7f6b2c159040 2024-08-15T18:01:40.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b2bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b2c15cea0 0x7f6b2c157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b2bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b2c15cea0 0x7f6b2c157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44024/0 (socket says 172.21.15.38:44024) 2024-08-15T18:01:40.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b2bfff700 1 -- 172.21.15.38:0/3270704472 learned_addr learned my addr 172.21.15.38:0/3270704472 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:40.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b23fff700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b2c159040 0x7f6b2c0b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b2b7fe700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b2c0d0010 0x7f6b2c151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b2bfff700 1 -- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b2c159040 msgr2=0x7f6b2c0b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b2bfff700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b2c159040 0x7f6b2c0b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b2bfff700 1 -- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b2c0d0010 msgr2=0x7f6b2c151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b2bfff700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b2c0d0010 0x7f6b2c151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b2bfff700 1 -- 172.21.15.38:0/3270704472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b24009ae0 con 0x7f6b2c15cea0 2024-08-15T18:01:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b2b7fe700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b2c0d0010 0x7f6b2c151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.914+0000 7f6b2bfff700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b2c15cea0 0x7f6b2c157ce0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009fd0 tx=0x7f6b24018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.918+0000 7f6b237fe700 1 -- 172.21.15.38:0/3270704472 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400be80 con 0x7f6b2c15cea0 2024-08-15T18:01:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.918+0000 7f6b237fe700 1 -- 172.21.15.38:0/3270704472 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24007df0 con 0x7f6b2c15cea0 2024-08-15T18:01:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.918+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3270704472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c0b21a0 con 0x7f6b2c15cea0 2024-08-15T18:01:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.918+0000 7f6b237fe700 1 -- 172.21.15.38:0/3270704472 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2401c660 con 0x7f6b2c15cea0 2024-08-15T18:01:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.918+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3270704472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c151db0 con 0x7f6b2c15cea0 2024-08-15T18:01:40.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.918+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3270704472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c0643a0 con 0x7f6b2c15cea0 2024-08-15T18:01:40.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.918+0000 7f6b237fe700 1 -- 172.21.15.38:0/3270704472 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6b24003050 con 0x7f6b2c15cea0 2024-08-15T18:01:40.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.918+0000 7f6b237fe700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b10060860 0x7f6b10062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:40.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.918+0000 7f6b237fe700 1 -- 172.21.15.38:0/3270704472 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f6b2407e950 con 0x7f6b2c15cea0 2024-08-15T18:01:40.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.918+0000 7f6b2b7fe700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b10060860 0x7f6b10062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:40.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.922+0000 7f6b2b7fe700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b10060860 0x7f6b10062d10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c007fd0 tx=0x7f6b1c007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:40.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:40.926+0000 7f6b237fe700 1 -- 172.21.15.38:0/3270704472 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6b2404ea20 con 0x7f6b2c15cea0 2024-08-15T18:01:41.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.126+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3270704472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b2c154670 con 0x7f6b2c15cea0 2024-08-15T18:01:41.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.126+0000 7f6b237fe700 1 -- 172.21.15.38:0/3270704472 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5681 (secure 0 0 0) 0x7f6b24052040 con 0x7f6b2c15cea0 2024-08-15T18:01:41.131 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:41.131 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/373633111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":373633111},{"type":"v1","addr":"172.21.15.38:6833","nonce":373633111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/389886189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":389886189},{"type":"v1","addr":"172.21.15.38:6835","nonce":389886189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:20.353438+0000","modified":"2024-08-15T17:59:24.015140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4999},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4999":{"gid":4999,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/1865762791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1865762791},{"type":"v1","addr":"172.21.15.104:6837","nonce":1865762791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:35.121185+0000","modified":"2024-08-15T17:59:36.892161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4981":{"gid":4981,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6835/875828080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":875828080},{"type":"v1","addr":"172.21.15.104:6835","nonce":875828080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000845064","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T18:01:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b10060860 msgr2=0x7f6b10062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b10060860 0x7f6b10062d10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c007fd0 tx=0x7f6b1c007e30 comp rx=0 tx=0).stop 2024-08-15T18:01:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3270704472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b2c15cea0 msgr2=0x7f6b2c157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b2c15cea0 0x7f6b2c157ce0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009fd0 tx=0x7f6b24018000 comp rx=0 tx=0).stop 2024-08-15T18:01:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3270704472 shutdown_connections 2024-08-15T18:01:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6b10060860 0x7f6b10062d10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b2c15cea0 0x7f6b2c157ce0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b2c159040 0x7f6b2c0b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 --2- 172.21.15.38:0/3270704472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b2c0d0010 0x7f6b2c151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3270704472 >> 172.21.15.38:0/3270704472 conn(0x7f6b2c068250 msgr2=0x7f6b2c0b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3270704472 shutdown_connections 2024-08-15T18:01:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.130+0000 7f6b30bc5700 1 -- 172.21.15.38:0/3270704472 wait complete. 2024-08-15T18:01:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 18 2024-08-15T18:01:41.145 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T18:01:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.250+0000 7fb8f2cfb700 1 Processor -- start 2024-08-15T18:01:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.250+0000 7fb8f2cfb700 1 -- start start 2024-08-15T18:01:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f2cfb700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8ec159040 0x7fb8ec0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f2cfb700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec0cf190 con 0x7fb8ec15d5b0 2024-08-15T18:01:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f2cfb700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec0ce5e0 con 0x7fb8ec0d0010 2024-08-15T18:01:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f2cfb700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0ce860 con 0x7fb8ec159040 2024-08-15T18:01:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f24fa700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb8ec15d5b0 0x7fb8ec15aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50540/0 (socket says 172.21.15.38:50540) 2024-08-15T18:01:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f24fa700 1 -- 172.21.15.38:0/3828036686 learned_addr learned my addr 172.21.15.38:0/3828036686 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f1cf9700 1 --2- 172.21.15.38:0/3828036686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8ec159040 0x7fb8ec0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f0cf7700 1 -- 172.21.15.38:0/3828036686 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966055239 0 0) 0x7fb8ec0cf190 con 0x7fb8ec15d5b0 2024-08-15T18:01:41.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f0cf7700 1 -- 172.21.15.38:0/3828036686 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8d8003570 con 0x7fb8ec15d5b0 2024-08-15T18:01:41.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f0cf7700 1 -- 172.21.15.38:0/3828036686 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080436598 0 0) 0x7fb8ec0ce5e0 con 0x7fb8ec0d0010 2024-08-15T18:01:41.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f0cf7700 1 -- 172.21.15.38:0/3828036686 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8ec0cf190 con 0x7fb8ec0d0010 2024-08-15T18:01:41.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f1cf9700 1 -- 172.21.15.38:0/3828036686 >> v1:172.21.15.104:6789/0 conn(0x7fb8ec0d0010 legacy=0x7fb8ec15cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f1cf9700 1 -- 172.21.15.38:0/3828036686 >> v1:172.21.15.38:6789/0 conn(0x7fb8ec15d5b0 legacy=0x7fb8ec15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f1cf9700 1 -- 172.21.15.38:0/3828036686 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec0ceb40 con 0x7fb8ec159040 2024-08-15T18:01:41.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f1cf9700 1 --2- 172.21.15.38:0/3828036686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8ec159040 0x7fb8ec0cf620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009910 tx=0x7fb8e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9241c950f10b05fb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:41.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f0cf7700 1 -- 172.21.15.38:0/3828036686 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8011630 con 0x7fb8ec159040 2024-08-15T18:01:41.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f0cf7700 1 -- 172.21.15.38:0/3828036686 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e8011790 con 0x7fb8ec159040 2024-08-15T18:01:41.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f0cf7700 1 -- 172.21.15.38:0/3828036686 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8011b10 con 0x7fb8ec159040 2024-08-15T18:01:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/3828036686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8ec159040 msgr2=0x7fb8ec0cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f2cfb700 1 --2- 172.21.15.38:0/3828036686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8ec159040 0x7fb8ec0cf620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009910 tx=0x7fb8e80091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/3828036686 shutdown_connections 2024-08-15T18:01:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f2cfb700 1 --2- 172.21.15.38:0/3828036686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8ec159040 0x7fb8ec0cf620 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/3828036686 >> 172.21.15.38:0/3828036686 conn(0x7fb8ec068250 msgr2=0x7fb8ec068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/3828036686 shutdown_connections 2024-08-15T18:01:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.254+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/3828036686 wait complete. 2024-08-15T18:01:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f2cfb700 1 Processor -- start 2024-08-15T18:01:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f2cfb700 1 -- start start 2024-08-15T18:01:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f2cfb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec0d0010 0x7fb8ec149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f2cfb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8ec159040 0x7fb8ec15a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f2cfb700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8ec15d5b0 0x7fb8ec0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f2cfb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec14a0f0 con 0x7fb8ec0d0010 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f2cfb700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0b2740 con 0x7fb8ec159040 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f2cfb700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0b0fd0 con 0x7fb8ec15d5b0 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f1cf9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec0d0010 0x7fb8ec149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f1cf9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec0d0010 0x7fb8ec149bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44036/0 (socket says 172.21.15.38:44036) 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f1cf9700 1 -- 172.21.15.38:0/1024383958 learned_addr learned my addr 172.21.15.38:0/1024383958 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f24fa700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8ec15d5b0 0x7fb8ec0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.258+0000 7fb8f14f8700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8ec159040 0x7fb8ec15a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8f1cf9700 1 -- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8ec15d5b0 msgr2=0x7fb8ec0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8f1cf9700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8ec15d5b0 0x7fb8ec0b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8f1cf9700 1 -- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8ec159040 msgr2=0x7fb8ec15a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8f1cf9700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8ec159040 0x7fb8ec15a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8f1cf9700 1 -- 172.21.15.38:0/1024383958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e8009420 con 0x7fb8ec0d0010 2024-08-15T18:01:41.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8f1cf9700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec0d0010 0x7fb8ec149bb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8013000 tx=0x7fb8e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8e1ffb700 1 -- 172.21.15.38:0/1024383958 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e801f070 con 0x7fb8ec0d0010 2024-08-15T18:01:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8e1ffb700 1 -- 172.21.15.38:0/1024383958 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e8002f30 con 0x7fb8ec0d0010 2024-08-15T18:01:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/1024383958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec14be90 con 0x7fb8ec0d0010 2024-08-15T18:01:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8e1ffb700 1 -- 172.21.15.38:0/1024383958 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8011900 con 0x7fb8ec0d0010 2024-08-15T18:01:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/1024383958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec14c450 con 0x7fb8ec0d0010 2024-08-15T18:01:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.262+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/1024383958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec0643a0 con 0x7fb8ec0d0010 2024-08-15T18:01:41.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.266+0000 7fb8e1ffb700 1 -- 172.21.15.38:0/1024383958 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb8e8023070 con 0x7fb8ec0d0010 2024-08-15T18:01:41.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.266+0000 7fb8e1ffb700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb8d0060860 0x7fb8d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.266+0000 7fb8e1ffb700 1 -- 172.21.15.38:0/1024383958 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fb8e807e1c0 con 0x7fb8ec0d0010 2024-08-15T18:01:41.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.266+0000 7fb8f14f8700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb8d0060860 0x7fb8d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.266+0000 7fb8f14f8700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb8d0060860 0x7fb8d0062d10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec15c820 tx=0x7fb8dc007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:41.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.274+0000 7fb8e1ffb700 1 -- 172.21.15.38:0/1024383958 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb8e804e290 con 0x7fb8ec0d0010 2024-08-15T18:01:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.438+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/1024383958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8ec0b3480 con 0x7fb8ec0d0010 2024-08-15T18:01:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8e1ffb700 1 -- 172.21.15.38:0/1024383958 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18599 (secure 0 0 0) 0x7fb8e8006cb0 con 0x7fb8ec0d0010 2024-08-15T18:01:41.444 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:41.444 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":37,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T17:59:38.114973+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:59:18.793510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:59:19.328139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000845064.meta","create_time":"2024-08-15T17:59:33.132275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000845064"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000845064.data","create_time":"2024-08-15T17:59:34.143101+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000845064"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T18:01:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb8d0060860 msgr2=0x7fb8d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb8d0060860 0x7fb8d0062d10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec15c820 tx=0x7fb8dc007d30 comp rx=0 tx=0).stop 2024-08-15T18:01:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/1024383958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec0d0010 msgr2=0x7fb8ec149bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec0d0010 0x7fb8ec149bb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8013000 tx=0x7fb8e8017000 comp rx=0 tx=0).stop 2024-08-15T18:01:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/1024383958 shutdown_connections 2024-08-15T18:01:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb8d0060860 0x7fb8d0062d10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8ec15d5b0 0x7fb8ec0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8ec159040 0x7fb8ec15a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 --2- 172.21.15.38:0/1024383958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec0d0010 0x7fb8ec149bb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/1024383958 >> 172.21.15.38:0/1024383958 conn(0x7fb8ec068250 msgr2=0x7fb8ec1580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/1024383958 shutdown_connections 2024-08-15T18:01:41.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.442+0000 7fb8f2cfb700 1 -- 172.21.15.38:0/1024383958 wait complete. 2024-08-15T18:01:41.459 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-08-15T18:01:41.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.570+0000 7f06c2151700 1 Processor -- start 2024-08-15T18:01:41.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.570+0000 7f06c2151700 1 -- start start 2024-08-15T18:01:41.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.570+0000 7f06c2151700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06bc0b0d20 0x7f06bc06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.570+0000 7f06c2151700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f06bc06e450 0x7f06bc0b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.570+0000 7f06c2151700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06bc141a00 con 0x7f06bc0b1630 2024-08-15T18:01:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.570+0000 7f06c2151700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc06e8a0 con 0x7f06bc06e450 2024-08-15T18:01:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.570+0000 7f06c2151700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc140b10 con 0x7f06bc0b0d20 2024-08-15T18:01:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.570+0000 7f06c094e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f06bc06e450 0x7f06bc0b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.570+0000 7f06c094e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f06bc06e450 0x7f06bc0b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44046/0 (socket says 172.21.15.38:44046) 2024-08-15T18:01:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.570+0000 7f06c094e700 1 -- 172.21.15.38:0/3266334597 learned_addr learned my addr 172.21.15.38:0/3266334597 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.570+0000 7f06c114f700 1 --2- 172.21.15.38:0/3266334597 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06bc0b0d20 0x7f06bc06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c094e700 1 -- 172.21.15.38:0/3266334597 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06bc0b0d20 msgr2=0x7f06bc06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c094e700 1 --2- 172.21.15.38:0/3266334597 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06bc0b0d20 0x7f06bc06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c094e700 1 -- 172.21.15.38:0/3266334597 >> v1:172.21.15.104:6789/0 conn(0x7f06bc0b1630 legacy=0x7f06bc1610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c094e700 1 -- 172.21.15.38:0/3266334597 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc140df0 con 0x7f06bc06e450 2024-08-15T18:01:41.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c094e700 1 --2- 172.21.15.38:0/3266334597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06bc06e450 0x7f06bc0b10f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009f20 tx=0x7f06b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609b3be5dc876c2d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:41.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06affff700 1 -- 172.21.15.38:0/3266334597 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b000e070 con 0x7f06bc06e450 2024-08-15T18:01:41.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06affff700 1 -- 172.21.15.38:0/3266334597 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b000b9d0 con 0x7f06bc06e450 2024-08-15T18:01:41.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06affff700 1 -- 172.21.15.38:0/3266334597 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b00092e0 con 0x7f06bc06e450 2024-08-15T18:01:41.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c2151700 1 -- 172.21.15.38:0/3266334597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06bc06e450 msgr2=0x7f06bc0b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c2151700 1 --2- 172.21.15.38:0/3266334597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06bc06e450 0x7f06bc0b10f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009f20 tx=0x7f06b0009840 comp rx=0 tx=0).stop 2024-08-15T18:01:41.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c2151700 1 -- 172.21.15.38:0/3266334597 shutdown_connections 2024-08-15T18:01:41.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c2151700 1 --2- 172.21.15.38:0/3266334597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06bc06e450 0x7f06bc0b10f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c2151700 1 --2- 172.21.15.38:0/3266334597 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06bc0b0d20 0x7f06bc06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c2151700 1 -- 172.21.15.38:0/3266334597 >> 172.21.15.38:0/3266334597 conn(0x7f06bc0678d0 msgr2=0x7f06bc067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:41.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c2151700 1 -- 172.21.15.38:0/3266334597 shutdown_connections 2024-08-15T18:01:41.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.574+0000 7f06c2151700 1 -- 172.21.15.38:0/3266334597 wait complete. 2024-08-15T18:01:41.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c2151700 1 Processor -- start 2024-08-15T18:01:41.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c2151700 1 -- start start 2024-08-15T18:01:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c2151700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06bc06e450 0x7f06bc13fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c2151700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06bc0b0d20 0x7f06bc145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c2151700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06bc0b1630 0x7f06bc1462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c2151700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc151800 con 0x7f06bc0b0d20 2024-08-15T18:01:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c2151700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc150bf0 con 0x7f06bc0b1630 2024-08-15T18:01:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c2151700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc150ed0 con 0x7f06bc06e450 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c094e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06bc0b0d20 0x7f06bc145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c094e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06bc0b0d20 0x7f06bc145d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44052/0 (socket says 172.21.15.38:44052) 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c094e700 1 -- 172.21.15.38:0/1160817406 learned_addr learned my addr 172.21.15.38:0/1160817406 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c1950700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06bc0b1630 0x7f06bc1462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c114f700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06bc06e450 0x7f06bc13fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c094e700 1 -- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06bc06e450 msgr2=0x7f06bc13fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c094e700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06bc06e450 0x7f06bc13fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c094e700 1 -- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06bc0b1630 msgr2=0x7f06bc1462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c094e700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06bc0b1630 0x7f06bc1462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c094e700 1 -- 172.21.15.38:0/1160817406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b0009ae0 con 0x7f06bc0b0d20 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c094e700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06bc0b0d20 0x7f06bc145d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b610 tx=0x7f06b0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06ad7fa700 1 -- 172.21.15.38:0/1160817406 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b00089e0 con 0x7f06bc0b0d20 2024-08-15T18:01:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06ad7fa700 1 -- 172.21.15.38:0/1160817406 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b0008b60 con 0x7f06bc0b0d20 2024-08-15T18:01:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c2151700 1 -- 172.21.15.38:0/1160817406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc1511b0 con 0x7f06bc0b0d20 2024-08-15T18:01:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06ad7fa700 1 -- 172.21.15.38:0/1160817406 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0026460 con 0x7f06bc0b0d20 2024-08-15T18:01:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.578+0000 7f06c2151700 1 -- 172.21.15.38:0/1160817406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc140420 con 0x7f06bc0b0d20 2024-08-15T18:01:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.582+0000 7f06c2151700 1 -- 172.21.15.38:0/1160817406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06bc1519b0 con 0x7f06bc0b0d20 2024-08-15T18:01:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.582+0000 7f06ad7fa700 1 -- 172.21.15.38:0/1160817406 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f06b0003040 con 0x7f06bc0b0d20 2024-08-15T18:01:41.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.582+0000 7f06ad7fa700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06a0060860 0x7f06a0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.582+0000 7f06ad7fa700 1 -- 172.21.15.38:0/1160817406 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f06b007ec40 con 0x7f06bc0b0d20 2024-08-15T18:01:41.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.582+0000 7f06c114f700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06a0060860 0x7f06a0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.582+0000 7f06c114f700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06a0060860 0x7f06a0062d10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009db0 tx=0x7f06b80091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:41.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.590+0000 7f06ad7fa700 1 -- 172.21.15.38:0/1160817406 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f06bc1519b0 con 0x7f06bc0b0d20 2024-08-15T18:01:41.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.790+0000 7f06c2151700 1 -- 172.21.15.38:0/1160817406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f06bc0643a0 con 0x7f06bc0b0d20 2024-08-15T18:01:41.795 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-08-15T18:01:41.794+0000 7fa9f6684700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-15T18:01:41.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06ad7fa700 1 -- 172.21.15.38:0/1160817406 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f06b004ec70 con 0x7f06bc0b0d20 2024-08-15T18:01:41.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 -- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06a0060860 msgr2=0x7f06a0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06a0060860 0x7f06a0062d10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009db0 tx=0x7f06b80091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 -- 172.21.15.38:0/1160817406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06bc0b0d20 msgr2=0x7f06bc145d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06bc0b0d20 0x7f06bc145d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b610 tx=0x7f06b0018000 comp rx=0 tx=0).stop 2024-08-15T18:01:41.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 -- 172.21.15.38:0/1160817406 shutdown_connections 2024-08-15T18:01:41.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f06a0060860 0x7f06a0062d10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06bc0b1630 0x7f06bc1462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06bc0b0d20 0x7f06bc145d70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 --2- 172.21.15.38:0/1160817406 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06bc06e450 0x7f06bc13fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 -- 172.21.15.38:0/1160817406 >> 172.21.15.38:0/1160817406 conn(0x7f06bc0678d0 msgr2=0x7f06bc068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:41.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 -- 172.21.15.38:0/1160817406 shutdown_connections 2024-08-15T18:01:41.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.802+0000 7f06c2151700 1 -- 172.21.15.38:0/1160817406 wait complete. 2024-08-15T18:01:41.808 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-15T18:01:41.812 INFO:tasks.ceph.mds.d.smithi104.stderr: -7271> 2024-08-15T17:59:24.814+0000 7f199d96b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T18:01:41.820 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-08-15T18:01:41.841 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-08-15T18:01:41.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.930+0000 7f126dc02700 1 Processor -- start 2024-08-15T18:01:41.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.930+0000 7f126dc02700 1 -- start start 2024-08-15T18:01:41.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.930+0000 7f126dc02700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f126815d4c0 0x7f126806e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.930+0000 7f126dc02700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f12681611c0 0x7f12680686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.930+0000 7f126dc02700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1268068fe0 con 0x7f1268068be0 2024-08-15T18:01:41.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.930+0000 7f126dc02700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1268161930 con 0x7f126815d4c0 2024-08-15T18:01:41.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.930+0000 7f126dc02700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1268161c10 con 0x7f12681611c0 2024-08-15T18:01:41.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126cc00700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f126815d4c0 0x7f126806e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126d401700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1268068be0 0x7f12680ae040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50552/0 (socket says 172.21.15.38:50552) 2024-08-15T18:01:41.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126d401700 1 -- 172.21.15.38:0/1477874386 learned_addr learned my addr 172.21.15.38:0/1477874386 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:41.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f1267fff700 1 --2- 172.21.15.38:0/1477874386 >> v2:172.21.15.104:3300/0 conn(0x7f12681611c0 0x7f12680686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126cc00700 1 -- 172.21.15.38:0/1477874386 >> v2:172.21.15.104:3300/0 conn(0x7f12681611c0 msgr2=0x7f12680686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:41.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126cc00700 1 --2- 172.21.15.38:0/1477874386 >> v2:172.21.15.104:3300/0 conn(0x7f12681611c0 0x7f12680686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126cc00700 1 -- 172.21.15.38:0/1477874386 >> v1:172.21.15.38:6789/0 conn(0x7f1268068be0 legacy=0x7f12680ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126cc00700 1 -- 172.21.15.38:0/1477874386 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1268161ef0 con 0x7f126815d4c0 2024-08-15T18:01:41.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f12677fe700 1 -- 172.21.15.38:0/1477874386 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608526854 0 0) 0x7f1268068fe0 con 0x7f1268068be0 2024-08-15T18:01:41.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126cc00700 1 --2- 172.21.15.38:0/1477874386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f126815d4c0 0x7f126806e710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f126806c990 tx=0x7f12600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90977b33500b25 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:41.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f12677fe700 1 -- 172.21.15.38:0/1477874386 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260011630 con 0x7f126815d4c0 2024-08-15T18:01:41.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f12677fe700 1 -- 172.21.15.38:0/1477874386 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1260011790 con 0x7f126815d4c0 2024-08-15T18:01:41.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f12677fe700 1 -- 172.21.15.38:0/1477874386 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260011b30 con 0x7f126815d4c0 2024-08-15T18:01:41.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126dc02700 1 -- 172.21.15.38:0/1477874386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f126815d4c0 msgr2=0x7f126806e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126dc02700 1 --2- 172.21.15.38:0/1477874386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f126815d4c0 0x7f126806e710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f126806c990 tx=0x7f12600091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126dc02700 1 -- 172.21.15.38:0/1477874386 shutdown_connections 2024-08-15T18:01:41.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126dc02700 1 --2- 172.21.15.38:0/1477874386 >> v2:172.21.15.104:3300/0 conn(0x7f12681611c0 0x7f12680686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126dc02700 1 --2- 172.21.15.38:0/1477874386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f126815d4c0 0x7f126806e710 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126dc02700 1 -- 172.21.15.38:0/1477874386 >> 172.21.15.38:0/1477874386 conn(0x7f12680ccfc0 msgr2=0x7f126806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:41.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126dc02700 1 -- 172.21.15.38:0/1477874386 shutdown_connections 2024-08-15T18:01:41.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126dc02700 1 -- 172.21.15.38:0/1477874386 wait complete. 2024-08-15T18:01:41.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126dc02700 1 Processor -- start 2024-08-15T18:01:41.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.934+0000 7f126dc02700 1 -- start start 2024-08-15T18:01:41.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126dc02700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1268068be0 0x7f12681580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126dc02700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f126815d4c0 0x7f126815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126dc02700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12681611c0 0x7f12681543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126dc02700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126815eb30 con 0x7f12681611c0 2024-08-15T18:01:41.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126dc02700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12680b1430 con 0x7f126815d4c0 2024-08-15T18:01:41.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126dc02700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12680b1710 con 0x7f1268068be0 2024-08-15T18:01:41.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126d401700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12681611c0 0x7f12681543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126d401700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12681611c0 0x7f12681543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44082/0 (socket says 172.21.15.38:44082) 2024-08-15T18:01:41.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126d401700 1 -- 172.21.15.38:0/184911792 learned_addr learned my addr 172.21.15.38:0/184911792 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:41.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126cc00700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1268068be0 0x7f12681580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f1267fff700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f126815d4c0 0x7f126815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126d401700 1 -- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1268068be0 msgr2=0x7f12681580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126d401700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1268068be0 0x7f12681580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126d401700 1 -- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f126815d4c0 msgr2=0x7f126815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:41.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126d401700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f126815d4c0 0x7f126815c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:41.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126d401700 1 -- 172.21.15.38:0/184911792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1260009420 con 0x7f12681611c0 2024-08-15T18:01:41.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126d401700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12681611c0 0x7f12681543b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f12580011e0 tx=0x7f1258007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:41.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f1264ff9700 1 -- 172.21.15.38:0/184911792 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258005b30 con 0x7f12681611c0 2024-08-15T18:01:41.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f1264ff9700 1 -- 172.21.15.38:0/184911792 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1258005c90 con 0x7f12681611c0 2024-08-15T18:01:41.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126dc02700 1 -- 172.21.15.38:0/184911792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1268155fa0 con 0x7f12681611c0 2024-08-15T18:01:41.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f1264ff9700 1 -- 172.21.15.38:0/184911792 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258007320 con 0x7f12681611c0 2024-08-15T18:01:41.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126dc02700 1 -- 172.21.15.38:0/184911792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1268156560 con 0x7f12681611c0 2024-08-15T18:01:41.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f126dc02700 1 -- 172.21.15.38:0/184911792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1268156750 con 0x7f12681611c0 2024-08-15T18:01:41.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.938+0000 7f1264ff9700 1 -- 172.21.15.38:0/184911792 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1258007690 con 0x7f12681611c0 2024-08-15T18:01:41.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.942+0000 7f1264ff9700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1248060860 0x7f1248062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:41.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.942+0000 7f1264ff9700 1 -- 172.21.15.38:0/184911792 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f125807c450 con 0x7f12681611c0 2024-08-15T18:01:41.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.942+0000 7f126cc00700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1248060860 0x7f1248062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:41.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.942+0000 7f126cc00700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1248060860 0x7f1248062d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12600093f0 tx=0x7f1260017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:41.946+0000 7f1264ff9700 1 -- 172.21.15.38:0/184911792 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f125804c430 con 0x7f12681611c0 2024-08-15T18:01:42.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.146+0000 7f126dc02700 1 -- 172.21.15.38:0/184911792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f12680638f0 con 0x7f12681611c0 2024-08-15T18:01:42.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.806+0000 7f1264ff9700 1 -- 172.21.15.38:0/184911792 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f125804fa50 con 0x7f12681611c0 2024-08-15T18:01:42.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.810+0000 7f126dc02700 1 -- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1248060860 msgr2=0x7f1248062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:42.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.810+0000 7f126dc02700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1248060860 0x7f1248062d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12600093f0 tx=0x7f1260017000 comp rx=0 tx=0).stop 2024-08-15T18:01:42.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.814+0000 7f126dc02700 1 -- 172.21.15.38:0/184911792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12681611c0 msgr2=0x7f12681543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:42.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.814+0000 7f126dc02700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12681611c0 0x7f12681543b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f12580011e0 tx=0x7f1258007d30 comp rx=0 tx=0).stop 2024-08-15T18:01:42.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.814+0000 7f126dc02700 1 -- 172.21.15.38:0/184911792 shutdown_connections 2024-08-15T18:01:42.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.814+0000 7f126dc02700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1248060860 0x7f1248062d10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:42.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.814+0000 7f126dc02700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12681611c0 0x7f12681543b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:42.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.814+0000 7f126dc02700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f126815d4c0 0x7f126815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:42.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.814+0000 7f126dc02700 1 --2- 172.21.15.38:0/184911792 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1268068be0 0x7f12681580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:42.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.814+0000 7f126dc02700 1 -- 172.21.15.38:0/184911792 >> 172.21.15.38:0/184911792 conn(0x7f12680ccfc0 msgr2=0x7f1268160730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:42.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.814+0000 7f126dc02700 1 -- 172.21.15.38:0/184911792 shutdown_connections 2024-08-15T18:01:42.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.814+0000 7f126dc02700 1 -- 172.21.15.38:0/184911792 wait complete. 2024-08-15T18:01:42.832 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-15T18:01:42.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a7a56c700 1 Processor -- start 2024-08-15T18:01:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a7a56c700 1 -- start start 2024-08-15T18:01:42.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a7a56c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a740c6160 0x7f7a740caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:42.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a7a56c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7a740cb3f0 0x7f7a740ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:42.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a7a56c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a740cc480 con 0x7f7a740cd2b0 2024-08-15T18:01:42.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a7a56c700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a74159720 con 0x7f7a740cb3f0 2024-08-15T18:01:42.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a7a56c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a740cb870 con 0x7f7a740c6160 2024-08-15T18:01:42.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a73fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7a740cd2b0 0x7f7a740cd680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50558/0 (socket says 172.21.15.38:50558) 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a73fff700 1 -- 172.21.15.38:0/2841601024 learned_addr learned my addr 172.21.15.38:0/2841601024 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a78b09700 1 --2- 172.21.15.38:0/2841601024 >> v2:172.21.15.38:3300/0 conn(0x7f7a740cb3f0 0x7f7a740ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a737fe700 1 --2- 172.21.15.38:0/2841601024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a740c6160 0x7f7a740caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a72ffd700 1 -- 172.21.15.38:0/2841601024 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658638992 0 0) 0x7f7a740cc480 con 0x7f7a740cd2b0 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a72ffd700 1 -- 172.21.15.38:0/2841601024 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a5c003570 con 0x7f7a740cd2b0 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a78b09700 1 -- 172.21.15.38:0/2841601024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a740c6160 msgr2=0x7f7a740caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a78b09700 1 --2- 172.21.15.38:0/2841601024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a740c6160 0x7f7a740caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a78b09700 1 -- 172.21.15.38:0/2841601024 >> v1:172.21.15.38:6789/0 conn(0x7f7a740cd2b0 legacy=0x7f7a740cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a78b09700 1 -- 172.21.15.38:0/2841601024 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a740cbb50 con 0x7f7a740cb3f0 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a78b09700 1 --2- 172.21.15.38:0/2841601024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a740cb3f0 0x7f7a740ce460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009e60 tx=0x7f7a6c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45aed8009d4e35f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a72ffd700 1 -- 172.21.15.38:0/2841601024 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00e070 con 0x7f7a740cb3f0 2024-08-15T18:01:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a72ffd700 1 -- 172.21.15.38:0/2841601024 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6c00bab0 con 0x7f7a740cb3f0 2024-08-15T18:01:42.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.954+0000 7f7a72ffd700 1 -- 172.21.15.38:0/2841601024 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00be70 con 0x7f7a740cb3f0 2024-08-15T18:01:42.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 -- 172.21.15.38:0/2841601024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a740cb3f0 msgr2=0x7f7a740ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:42.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 --2- 172.21.15.38:0/2841601024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a740cb3f0 0x7f7a740ce460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009e60 tx=0x7f7a6c009780 comp rx=0 tx=0).stop 2024-08-15T18:01:42.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 -- 172.21.15.38:0/2841601024 shutdown_connections 2024-08-15T18:01:42.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 --2- 172.21.15.38:0/2841601024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a740cb3f0 0x7f7a740ce460 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:42.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 --2- 172.21.15.38:0/2841601024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a740c6160 0x7f7a740caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:42.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 -- 172.21.15.38:0/2841601024 >> 172.21.15.38:0/2841601024 conn(0x7f7a74099550 msgr2=0x7f7a74067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:42.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 -- 172.21.15.38:0/2841601024 shutdown_connections 2024-08-15T18:01:42.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 -- 172.21.15.38:0/2841601024 wait complete. 2024-08-15T18:01:42.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 Processor -- start 2024-08-15T18:01:42.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 -- start start 2024-08-15T18:01:42.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a740c6160 0x7f7a74069940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:42.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a740cd2b0 0x7f7a740b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:42.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a740b5780 0x7f7a74156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:42.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a740b3f80 con 0x7f7a740cd2b0 2024-08-15T18:01:42.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a740b3050 con 0x7f7a740b5780 2024-08-15T18:01:42.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.958+0000 7f7a7a56c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a740b3330 con 0x7f7a740c6160 2024-08-15T18:01:42.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a737fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a740cd2b0 0x7f7a740b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:42.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a737fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a740cd2b0 0x7f7a740b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44102/0 (socket says 172.21.15.38:44102) 2024-08-15T18:01:42.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a737fe700 1 -- 172.21.15.38:0/1670840428 learned_addr learned my addr 172.21.15.38:0/1670840428 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:42.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a78b09700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a740b5780 0x7f7a74156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:42.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a73fff700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a740c6160 0x7f7a74069940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:42.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a737fe700 1 -- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a740c6160 msgr2=0x7f7a74069940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:42.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a737fe700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a740c6160 0x7f7a74069940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:42.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a737fe700 1 -- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a740b5780 msgr2=0x7f7a74156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:42.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a737fe700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a740b5780 0x7f7a74156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:42.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a737fe700 1 -- 172.21.15.38:0/1670840428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c009a20 con 0x7f7a740cd2b0 2024-08-15T18:01:42.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a737fe700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a740cd2b0 0x7f7a740b3a40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800e930 tx=0x7f7a680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:42.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a5bfff700 1 -- 172.21.15.38:0/1670840428 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68013680 con 0x7f7a740cd2b0 2024-08-15T18:01:42.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a5bfff700 1 -- 172.21.15.38:0/1670840428 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a680137e0 con 0x7f7a740cd2b0 2024-08-15T18:01:42.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a5bfff700 1 -- 172.21.15.38:0/1670840428 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68013b80 con 0x7f7a740cd2b0 2024-08-15T18:01:42.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a7a56c700 1 -- 172.21.15.38:0/1670840428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a740b3610 con 0x7f7a740cd2b0 2024-08-15T18:01:42.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a7a56c700 1 -- 172.21.15.38:0/1670840428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a74069420 con 0x7f7a740cd2b0 2024-08-15T18:01:42.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a7a56c700 1 -- 172.21.15.38:0/1670840428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a740638f0 con 0x7f7a740cd2b0 2024-08-15T18:01:42.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.962+0000 7f7a5bfff700 1 -- 172.21.15.38:0/1670840428 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7a6800c3e0 con 0x7f7a740cd2b0 2024-08-15T18:01:42.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.966+0000 7f7a5bfff700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7a60060860 0x7f7a60062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:42.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.966+0000 7f7a5bfff700 1 -- 172.21.15.38:0/1670840428 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f7a6807d160 con 0x7f7a740cd2b0 2024-08-15T18:01:42.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.966+0000 7f7a73fff700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7a60060860 0x7f7a60062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:42.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.966+0000 7f7a73fff700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7a60060860 0x7f7a60062d10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a64007fd0 tx=0x7f7a64009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:42.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:42.970+0000 7f7a5bfff700 1 -- 172.21.15.38:0/1670840428 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7a6804d130 con 0x7f7a740cd2b0 2024-08-15T18:01:43.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.130+0000 7f7a7a56c700 1 -- 172.21.15.38:0/1670840428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7a74068020 con 0x7f7a740cd2b0 2024-08-15T18:01:43.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.814+0000 7f7a5bfff700 1 -- 172.21.15.38:0/1670840428 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7a68050750 con 0x7f7a740cd2b0 2024-08-15T18:01:43.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 -- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7a60060860 msgr2=0x7f7a60062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:43.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7a60060860 0x7f7a60062d10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a64007fd0 tx=0x7f7a64009040 comp rx=0 tx=0).stop 2024-08-15T18:01:43.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 -- 172.21.15.38:0/1670840428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a740cd2b0 msgr2=0x7f7a740b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:43.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a740cd2b0 0x7f7a740b3a40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800e930 tx=0x7f7a680091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:43.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 -- 172.21.15.38:0/1670840428 shutdown_connections 2024-08-15T18:01:43.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7a60060860 0x7f7a60062d10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:43.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a740b5780 0x7f7a74156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:43.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a740cd2b0 0x7f7a740b3a40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:43.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 --2- 172.21.15.38:0/1670840428 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a740c6160 0x7f7a74069940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:43.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 -- 172.21.15.38:0/1670840428 >> 172.21.15.38:0/1670840428 conn(0x7f7a74099550 msgr2=0x7f7a74067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:43.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 -- 172.21.15.38:0/1670840428 shutdown_connections 2024-08-15T18:01:43.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.818+0000 7f7a7a56c700 1 -- 172.21.15.38:0/1670840428 wait complete. 2024-08-15T18:01:43.823 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-08-15T18:01:43.837 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-15T18:01:43.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 Processor -- start 2024-08-15T18:01:43.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 -- start start 2024-08-15T18:01:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde900cd2b0 0x7fde900cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fde900ceb70 0x7fde90162ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde900cc480 con 0x7fde900caeb0 2024-08-15T18:01:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde90159720 con 0x7fde900ceb70 2024-08-15T18:01:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900cb870 con 0x7fde900cd2b0 2024-08-15T18:01:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde87fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fde900caeb0 0x7fde900ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50574/0 (socket says 172.21.15.38:50574) 2024-08-15T18:01:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde87fff700 1 -- 172.21.15.38:0/3476134970 learned_addr learned my addr 172.21.15.38:0/3476134970 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8ffff700 1 --2- 172.21.15.38:0/3476134970 >> v2:172.21.15.38:3300/0 conn(0x7fde900ceb70 0x7fde90162ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8f7fe700 1 --2- 172.21.15.38:0/3476134970 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde900cd2b0 0x7fde900cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8effd700 1 -- 172.21.15.38:0/3476134970 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063479361 0 0) 0x7fde900cc480 con 0x7fde900caeb0 2024-08-15T18:01:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8effd700 1 -- 172.21.15.38:0/3476134970 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde7c003570 con 0x7fde900caeb0 2024-08-15T18:01:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8ffff700 1 -- 172.21.15.38:0/3476134970 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde900cd2b0 msgr2=0x7fde900cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8ffff700 1 --2- 172.21.15.38:0/3476134970 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde900cd2b0 0x7fde900cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8ffff700 1 -- 172.21.15.38:0/3476134970 >> v1:172.21.15.38:6789/0 conn(0x7fde900caeb0 legacy=0x7fde900ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8ffff700 1 -- 172.21.15.38:0/3476134970 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde900cbb50 con 0x7fde900ceb70 2024-08-15T18:01:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8ffff700 1 --2- 172.21.15.38:0/3476134970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde900ceb70 0x7fde90162ea0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fde88009e60 tx=0x7fde88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d88f2d41ddb5c10 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8effd700 1 -- 172.21.15.38:0/3476134970 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8800e070 con 0x7fde900ceb70 2024-08-15T18:01:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8effd700 1 -- 172.21.15.38:0/3476134970 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8800bab0 con 0x7fde900ceb70 2024-08-15T18:01:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde8effd700 1 -- 172.21.15.38:0/3476134970 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8800be70 con 0x7fde900ceb70 2024-08-15T18:01:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 -- 172.21.15.38:0/3476134970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde900ceb70 msgr2=0x7fde90162ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 --2- 172.21.15.38:0/3476134970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde900ceb70 0x7fde90162ea0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fde88009e60 tx=0x7fde88009780 comp rx=0 tx=0).stop 2024-08-15T18:01:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 -- 172.21.15.38:0/3476134970 shutdown_connections 2024-08-15T18:01:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 --2- 172.21.15.38:0/3476134970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde900ceb70 0x7fde90162ea0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 --2- 172.21.15.38:0/3476134970 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde900cd2b0 0x7fde900cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.950+0000 7fde961aa700 1 -- 172.21.15.38:0/3476134970 >> 172.21.15.38:0/3476134970 conn(0x7fde90099550 msgr2=0x7fde90067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde961aa700 1 -- 172.21.15.38:0/3476134970 shutdown_connections 2024-08-15T18:01:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde961aa700 1 -- 172.21.15.38:0/3476134970 wait complete. 2024-08-15T18:01:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde961aa700 1 Processor -- start 2024-08-15T18:01:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde961aa700 1 -- start start 2024-08-15T18:01:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde961aa700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde900caeb0 0x7fde90155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde961aa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde900cd2b0 0x7fde900690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde961aa700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde900ceb70 0x7fde90069600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde961aa700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900c82c0 con 0x7fde900cd2b0 2024-08-15T18:01:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde961aa700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde90169820 con 0x7fde900ceb70 2024-08-15T18:01:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde961aa700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900c7740 con 0x7fde900caeb0 2024-08-15T18:01:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde87fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde900cd2b0 0x7fde900690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde87fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde900cd2b0 0x7fde900690c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44110/0 (socket says 172.21.15.38:44110) 2024-08-15T18:01:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde87fff700 1 -- 172.21.15.38:0/3674419496 learned_addr learned my addr 172.21.15.38:0/3674419496 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde8f7fe700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde900caeb0 0x7fde90155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.954+0000 7fde8ffff700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde900ceb70 0x7fde90069600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde87fff700 1 -- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde900caeb0 msgr2=0x7fde90155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde87fff700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde900caeb0 0x7fde90155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde87fff700 1 -- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde900ceb70 msgr2=0x7fde90069600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde87fff700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde900ceb70 0x7fde90069600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde87fff700 1 -- 172.21.15.38:0/3674419496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde88009a20 con 0x7fde900cd2b0 2024-08-15T18:01:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde87fff700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde900cd2b0 0x7fde900690c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fde7800c590 tx=0x7fde78007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde877fe700 1 -- 172.21.15.38:0/3674419496 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78005b30 con 0x7fde900cd2b0 2024-08-15T18:01:43.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde877fe700 1 -- 172.21.15.38:0/3674419496 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde78005cb0 con 0x7fde900cd2b0 2024-08-15T18:01:43.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde877fe700 1 -- 172.21.15.38:0/3674419496 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78007320 con 0x7fde900cd2b0 2024-08-15T18:01:43.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde961aa700 1 -- 172.21.15.38:0/3674419496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde900c7a20 con 0x7fde900cd2b0 2024-08-15T18:01:43.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde961aa700 1 -- 172.21.15.38:0/3674419496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde900c7fe0 con 0x7fde900cd2b0 2024-08-15T18:01:43.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde961aa700 1 -- 172.21.15.38:0/3674419496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde901699e0 con 0x7fde900cd2b0 2024-08-15T18:01:43.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde877fe700 1 -- 172.21.15.38:0/3674419496 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fde7801b3b0 con 0x7fde900cd2b0 2024-08-15T18:01:43.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde877fe700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fde70060860 0x7fde70062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:43.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.958+0000 7fde877fe700 1 -- 172.21.15.38:0/3674419496 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6251+0+0 (secure 0 0 0) 0x7fde7807c210 con 0x7fde900cd2b0 2024-08-15T18:01:43.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.962+0000 7fde8f7fe700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fde70060860 0x7fde70062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:43.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.962+0000 7fde8f7fe700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fde70060860 0x7fde70062d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde80009d10 tx=0x7fde800091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:43.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:43.966+0000 7fde877fe700 1 -- 172.21.15.38:0/3674419496 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fde901699e0 con 0x7fde900cd2b0 2024-08-15T18:01:44.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.126+0000 7fde961aa700 1 -- 172.21.15.38:0/3674419496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fde901699e0 con 0x7fde900cd2b0 2024-08-15T18:01:44.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.826+0000 7fde877fe700 1 -- 172.21.15.38:0/3674419496 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7fde901699e0 con 0x7fde900cd2b0 2024-08-15T18:01:44.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.830+0000 7fde961aa700 1 -- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fde70060860 msgr2=0x7fde70062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.830+0000 7fde961aa700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fde70060860 0x7fde70062d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde80009d10 tx=0x7fde800091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.830+0000 7fde961aa700 1 -- 172.21.15.38:0/3674419496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde900cd2b0 msgr2=0x7fde900690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.830+0000 7fde961aa700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde900cd2b0 0x7fde900690c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fde7800c590 tx=0x7fde78007e00 comp rx=0 tx=0).stop 2024-08-15T18:01:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.830+0000 7fde961aa700 1 -- 172.21.15.38:0/3674419496 shutdown_connections 2024-08-15T18:01:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.830+0000 7fde961aa700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fde70060860 0x7fde70062d10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.830+0000 7fde961aa700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde900ceb70 0x7fde90069600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.830+0000 7fde961aa700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde900cd2b0 0x7fde900690c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.834+0000 7fde961aa700 1 --2- 172.21.15.38:0/3674419496 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde900caeb0 0x7fde90155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.834+0000 7fde961aa700 1 -- 172.21.15.38:0/3674419496 >> 172.21.15.38:0/3674419496 conn(0x7fde90099550 msgr2=0x7fde90068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.834+0000 7fde961aa700 1 -- 172.21.15.38:0/3674419496 shutdown_connections 2024-08-15T18:01:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.834+0000 7fde961aa700 1 -- 172.21.15.38:0/3674419496 wait complete. 2024-08-15T18:01:44.841 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-08-15T18:01:44.865 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T18:01:44.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.962+0000 7fcde1c53700 1 Processor -- start 2024-08-15T18:01:44.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.962+0000 7fcde1c53700 1 -- start start 2024-08-15T18:01:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.962+0000 7fcde1c53700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcddc142780 0x7fcddc0ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.962+0000 7fcde1c53700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc0ca510 con 0x7fcddc0aef00 2024-08-15T18:01:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.962+0000 7fcde1c53700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc0c9900 con 0x7fcddc142b50 2024-08-15T18:01:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.962+0000 7fcde1c53700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc0c9be0 con 0x7fcddc142780 2024-08-15T18:01:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.962+0000 7fcde0c51700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcddc142780 0x7fcddc0ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.962+0000 7fcde1452700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcddc0aef00 0x7fcddc15a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50578/0 (socket says 172.21.15.38:50578) 2024-08-15T18:01:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.962+0000 7fcde1452700 1 -- 172.21.15.38:0/2368567008 learned_addr learned my addr 172.21.15.38:0/2368567008 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcde0c51700 1 -- 172.21.15.38:0/2368567008 >> v1:172.21.15.104:6789/0 conn(0x7fcddc142b50 legacy=0x7fcddc0ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcde0c51700 1 -- 172.21.15.38:0/2368567008 >> v1:172.21.15.38:6789/0 conn(0x7fcddc0aef00 legacy=0x7fcddc15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcddb7fe700 1 -- 172.21.15.38:0/2368567008 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742629392 0 0) 0x7fcddc0ca510 con 0x7fcddc0aef00 2024-08-15T18:01:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcde0c51700 1 -- 172.21.15.38:0/2368567008 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc0c9ec0 con 0x7fcddc142780 2024-08-15T18:01:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcde0c51700 1 --2- 172.21.15.38:0/2368567008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcddc142780 0x7fcddc0ae9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00098e0 tx=0x7fcdd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2df6eb6ea401882 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcddb7fe700 1 -- 172.21.15.38:0/2368567008 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0009e80 con 0x7fcddc142780 2024-08-15T18:01:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcddb7fe700 1 -- 172.21.15.38:0/2368567008 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd000b860 con 0x7fcddc142780 2024-08-15T18:01:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcddb7fe700 1 -- 172.21.15.38:0/2368567008 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000bbe0 con 0x7fcddc142780 2024-08-15T18:01:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcde1c53700 1 -- 172.21.15.38:0/2368567008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcddc142780 msgr2=0x7fcddc0ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcde1c53700 1 --2- 172.21.15.38:0/2368567008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcddc142780 0x7fcddc0ae9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00098e0 tx=0x7fcdd00091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcde1c53700 1 -- 172.21.15.38:0/2368567008 shutdown_connections 2024-08-15T18:01:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcde1c53700 1 --2- 172.21.15.38:0/2368567008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcddc142780 0x7fcddc0ae9c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcde1c53700 1 -- 172.21.15.38:0/2368567008 >> 172.21.15.38:0/2368567008 conn(0x7fcddc06e070 msgr2=0x7fcddc0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcde1c53700 1 -- 172.21.15.38:0/2368567008 shutdown_connections 2024-08-15T18:01:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.966+0000 7fcde1c53700 1 -- 172.21.15.38:0/2368567008 wait complete. 2024-08-15T18:01:44.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1c53700 1 Processor -- start 2024-08-15T18:01:44.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1c53700 1 -- start start 2024-08-15T18:01:44.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1c53700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcddc0aef00 0x7fcddc159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:44.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1c53700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcddc142780 0x7fcddc136da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:44.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1c53700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc142b50 0x7fcddc158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:44.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1c53700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc137860 con 0x7fcddc142780 2024-08-15T18:01:44.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1c53700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc137b40 con 0x7fcddc0aef00 2024-08-15T18:01:44.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1c53700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc137e20 con 0x7fcddc142b50 2024-08-15T18:01:44.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcddbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcddc142780 0x7fcddc136da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcddbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcddc142780 0x7fcddc136da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44122/0 (socket says 172.21.15.38:44122) 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcddbfff700 1 -- 172.21.15.38:0/704658425 learned_addr learned my addr 172.21.15.38:0/704658425 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde0c51700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcddc0aef00 0x7fcddc159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1452700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc142b50 0x7fcddc158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcddbfff700 1 -- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc142b50 msgr2=0x7fcddc158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcddbfff700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc142b50 0x7fcddc158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcddbfff700 1 -- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcddc0aef00 msgr2=0x7fcddc159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcddbfff700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcddc0aef00 0x7fcddc159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcddbfff700 1 -- 172.21.15.38:0/704658425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd0009420 con 0x7fcddc142780 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1452700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc142b50 0x7fcddc158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcddbfff700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcddc142780 0x7fcddc136da0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800c4e0 tx=0x7fcdc800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcdd8ff9700 1 -- 172.21.15.38:0/704658425 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8011590 con 0x7fcddc142780 2024-08-15T18:01:44.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcdd8ff9700 1 -- 172.21.15.38:0/704658425 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc8011710 con 0x7fcddc142780 2024-08-15T18:01:44.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1c53700 1 -- 172.21.15.38:0/704658425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc138100 con 0x7fcddc142780 2024-08-15T18:01:44.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcdd8ff9700 1 -- 172.21.15.38:0/704658425 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8011ad0 con 0x7fcddc142780 2024-08-15T18:01:44.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.970+0000 7fcde1c53700 1 -- 172.21.15.38:0/704658425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc13b120 con 0x7fcddc142780 2024-08-15T18:01:44.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.974+0000 7fcde1c53700 1 -- 172.21.15.38:0/704658425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc0638f0 con 0x7fcddc142780 2024-08-15T18:01:44.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.974+0000 7fcdd8ff9700 1 -- 172.21.15.38:0/704658425 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcdc801b3b0 con 0x7fcddc142780 2024-08-15T18:01:44.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.974+0000 7fcdd8ff9700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcdc0060890 0x7fcdc0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:44.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.974+0000 7fcdd8ff9700 1 -- 172.21.15.38:0/704658425 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fcdc807c150 con 0x7fcddc142780 2024-08-15T18:01:44.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.974+0000 7fcde0c51700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcdc0060890 0x7fcdc0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:44.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.974+0000 7fcde0c51700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcdc0060890 0x7fcdc0062d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00093f0 tx=0x7fcdd0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:44.982+0000 7fcdd8ff9700 1 -- 172.21.15.38:0/704658425 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcdc804c480 con 0x7fcddc142780 2024-08-15T18:01:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.146+0000 7fcde1c53700 1 -- 172.21.15.38:0/704658425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcddc0ae570 con 0x7fcddc142780 2024-08-15T18:01:45.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.150+0000 7fcdd8ff9700 1 -- 172.21.15.38:0/704658425 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+15514 (secure 0 0 0) 0x7fcdc804faa0 con 0x7fcddc142780 2024-08-15T18:01:45.153 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:45.153 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":40,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T18:01:44.812744+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000845064.meta","create_time":"2024-08-15T17:59:33.132275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000845064"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000845064.data","create_time":"2024-08-15T17:59:34.143101+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000845064"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:6837/1865762791":"2024-08-16T18:01:41.797233+0000","172.21.15.104:6836/1865762791":"2024-08-16T18:01:41.797233+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T18:01:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.150+0000 7fcde1c53700 1 -- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcdc0060890 msgr2=0x7fcdc0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.150+0000 7fcde1c53700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcdc0060890 0x7fcdc0062d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00093f0 tx=0x7fcdd0017000 comp rx=0 tx=0).stop 2024-08-15T18:01:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.150+0000 7fcde1c53700 1 -- 172.21.15.38:0/704658425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcddc142780 msgr2=0x7fcddc136da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.150+0000 7fcde1c53700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcddc142780 0x7fcddc136da0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800c4e0 tx=0x7fcdc800c7f0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.150+0000 7fcde1c53700 1 -- 172.21.15.38:0/704658425 shutdown_connections 2024-08-15T18:01:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.150+0000 7fcde1c53700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fcdc0060890 0x7fcdc0062d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.150+0000 7fcde1c53700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc142b50 0x7fcddc158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.150+0000 7fcde1c53700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcddc142780 0x7fcddc136da0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.150+0000 7fcde1c53700 1 --2- 172.21.15.38:0/704658425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcddc0aef00 0x7fcddc159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.150+0000 7fcde1c53700 1 -- 172.21.15.38:0/704658425 >> 172.21.15.38:0/704658425 conn(0x7fcddc06e070 msgr2=0x7fcddc15f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.154+0000 7fcde1c53700 1 -- 172.21.15.38:0/704658425 shutdown_connections 2024-08-15T18:01:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.154+0000 7fcde1c53700 1 -- 172.21.15.38:0/704658425 wait complete. 2024-08-15T18:01:45.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:58:22.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs.volume_0000000000845064.meta', 'create_time': '2024-08-15T17:59:33.132275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000845064'}}}, {'pool': 7, 'pool_name': 'cephfs.volume_0000000000845064.data', 'create_time': '2024-08-15T17:59:34.143101+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000845064'}}}] 2024-08-15T18:01:45.168 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T18:01:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.274+0000 7fe9535df700 1 Processor -- start 2024-08-15T18:01:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.274+0000 7fe9535df700 1 -- start start 2024-08-15T18:01:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe9535df700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe94c142c40 0x7fe94c067430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe9535df700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe94c067dd0 con 0x7fe94c067970 2024-08-15T18:01:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe9535df700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe94c15cde0 con 0x7fe94c142870 2024-08-15T18:01:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe9535df700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94c15d030 con 0x7fe94c142c40 2024-08-15T18:01:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe950b7a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe94c142c40 0x7fe94c067430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe951b7c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe94c067970 0x7fe94c15ab50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50584/0 (socket says 172.21.15.38:50584) 2024-08-15T18:01:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe951b7c700 1 -- 172.21.15.38:0/725732996 learned_addr learned my addr 172.21.15.38:0/725732996 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe950b7a700 1 -- 172.21.15.38:0/725732996 >> v1:172.21.15.104:6789/0 conn(0x7fe94c142870 legacy=0x7fe94c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe950b7a700 1 -- 172.21.15.38:0/725732996 >> v1:172.21.15.38:6789/0 conn(0x7fe94c067970 legacy=0x7fe94c15ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe94bfff700 1 -- 172.21.15.38:0/725732996 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413304110 0 0) 0x7fe944001110 con 0x7fe94c067970 2024-08-15T18:01:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe950b7a700 1 -- 172.21.15.38:0/725732996 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c15d310 con 0x7fe94c142c40 2024-08-15T18:01:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe950b7a700 1 --2- 172.21.15.38:0/725732996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94c142c40 0x7fe94c067430 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe9400098e0 tx=0x7fe9400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b581164f85e2115 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:45.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe94bfff700 1 -- 172.21.15.38:0/725732996 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940009e80 con 0x7fe94c142c40 2024-08-15T18:01:45.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe94bfff700 1 -- 172.21.15.38:0/725732996 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94000b860 con 0x7fe94c142c40 2024-08-15T18:01:45.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe94bfff700 1 -- 172.21.15.38:0/725732996 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94000bbe0 con 0x7fe94c142c40 2024-08-15T18:01:45.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe9535df700 1 -- 172.21.15.38:0/725732996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94c142c40 msgr2=0x7fe94c067430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe9535df700 1 --2- 172.21.15.38:0/725732996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94c142c40 0x7fe94c067430 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe9400098e0 tx=0x7fe9400091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe9535df700 1 -- 172.21.15.38:0/725732996 shutdown_connections 2024-08-15T18:01:45.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe9535df700 1 --2- 172.21.15.38:0/725732996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94c142c40 0x7fe94c067430 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe9535df700 1 -- 172.21.15.38:0/725732996 >> 172.21.15.38:0/725732996 conn(0x7fe94c0687a0 msgr2=0x7fe94c06c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:45.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe9535df700 1 -- 172.21.15.38:0/725732996 shutdown_connections 2024-08-15T18:01:45.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.278+0000 7fe9535df700 1 -- 172.21.15.38:0/725732996 wait complete. 2024-08-15T18:01:45.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe9535df700 1 Processor -- start 2024-08-15T18:01:45.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe9535df700 1 -- start start 2024-08-15T18:01:45.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe9535df700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe94c067970 0x7fe94c0ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe9535df700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe94c142870 0x7fe94c0c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe9535df700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94c142c40 0x7fe94c0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe9535df700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c169e60 con 0x7fe94c142c40 2024-08-15T18:01:45.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe9535df700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0c2450 con 0x7fe94c067970 2024-08-15T18:01:45.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe9535df700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c169b10 con 0x7fe94c142870 2024-08-15T18:01:45.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe951b7c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94c142c40 0x7fe94c0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe951b7c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94c142c40 0x7fe94c0cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44142/0 (socket says 172.21.15.38:44142) 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe951b7c700 1 -- 172.21.15.38:0/3777286549 learned_addr learned my addr 172.21.15.38:0/3777286549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe950b7a700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe94c142870 0x7fe94c0c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe95137b700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe94c067970 0x7fe94c0ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe951b7c700 1 -- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe94c142870 msgr2=0x7fe94c0c1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe951b7c700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe94c142870 0x7fe94c0c1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe951b7c700 1 -- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe94c067970 msgr2=0x7fe94c0ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe951b7c700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe94c067970 0x7fe94c0ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe951b7c700 1 -- 172.21.15.38:0/3777286549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe940009420 con 0x7fe94c142c40 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.282+0000 7fe95137b700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe94c067970 0x7fe94c0ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.286+0000 7fe951b7c700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94c142c40 0x7fe94c0cb8a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe94400c810 tx=0x7fe944007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.286+0000 7fe9497fa700 1 -- 172.21.15.38:0/3777286549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944005b30 con 0x7fe94c142c40 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.286+0000 7fe9497fa700 1 -- 172.21.15.38:0/3777286549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe944005cb0 con 0x7fe94c142c40 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.286+0000 7fe9535df700 1 -- 172.21.15.38:0/3777286549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c0c26f0 con 0x7fe94c142c40 2024-08-15T18:01:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.286+0000 7fe9497fa700 1 -- 172.21.15.38:0/3777286549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944004a80 con 0x7fe94c142c40 2024-08-15T18:01:45.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.286+0000 7fe9535df700 1 -- 172.21.15.38:0/3777286549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c0c2cb0 con 0x7fe94c142c40 2024-08-15T18:01:45.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.286+0000 7fe9535df700 1 -- 172.21.15.38:0/3777286549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94c169fd0 con 0x7fe94c142c40 2024-08-15T18:01:45.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.286+0000 7fe9497fa700 1 -- 172.21.15.38:0/3777286549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe944005030 con 0x7fe94c142c40 2024-08-15T18:01:45.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.286+0000 7fe9497fa700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe938060860 0x7fe938062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.286+0000 7fe9497fa700 1 -- 172.21.15.38:0/3777286549 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fe94407bfe0 con 0x7fe94c142c40 2024-08-15T18:01:45.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.286+0000 7fe95137b700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe938060860 0x7fe938062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.290+0000 7fe95137b700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe938060860 0x7fe938062d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe93c007fd0 tx=0x7fe93c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:45.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.294+0000 7fe9497fa700 1 -- 172.21.15.38:0/3777286549 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe94404c390 con 0x7fe94c142c40 2024-08-15T18:01:45.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.454+0000 7fe9535df700 1 -- 172.21.15.38:0/3777286549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe94c067dd0 con 0x7fe94c142c40 2024-08-15T18:01:45.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.454+0000 7fe9497fa700 1 -- 172.21.15.38:0/3777286549 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe94c067dd0 con 0x7fe94c142c40 2024-08-15T18:01:45.459 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-08-15T18:01:45.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 -- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe938060860 msgr2=0x7fe938062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe938060860 0x7fe938062d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe93c007fd0 tx=0x7fe93c009040 comp rx=0 tx=0).stop 2024-08-15T18:01:45.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 -- 172.21.15.38:0/3777286549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94c142c40 msgr2=0x7fe94c0cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94c142c40 0x7fe94c0cb8a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe94400c810 tx=0x7fe944007bd0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 -- 172.21.15.38:0/3777286549 shutdown_connections 2024-08-15T18:01:45.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe938060860 0x7fe938062d10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94c142c40 0x7fe94c0cb8a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe94c142870 0x7fe94c0c1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 --2- 172.21.15.38:0/3777286549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe94c067970 0x7fe94c0ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 -- 172.21.15.38:0/3777286549 >> 172.21.15.38:0/3777286549 conn(0x7fe94c0687a0 msgr2=0x7fe94c06a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:45.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 -- 172.21.15.38:0/3777286549 shutdown_connections 2024-08-15T18:01:45.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.458+0000 7fe9535df700 1 -- 172.21.15.38:0/3777286549 wait complete. 2024-08-15T18:01:45.473 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000845064.meta pg_num 2024-08-15T18:01:45.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.582+0000 7f82c7fff700 1 Processor -- start 2024-08-15T18:01:45.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.582+0000 7f82c7fff700 1 -- start start 2024-08-15T18:01:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82c8140790 0x7f82c80c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82c8140b60 0x7f82c80aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f82c806e700 0x7f82c80af100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c8067700 con 0x7f82c8140790 2024-08-15T18:01:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c8157a50 con 0x7f82c806e700 2024-08-15T18:01:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c8157700 con 0x7f82c8140b60 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c6ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82c8140790 0x7f82c80c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c6ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82c8140790 0x7f82c80c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44146/0 (socket says 172.21.15.38:44146) 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c77fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f82c806e700 0x7f82c80af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c67fc700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82c8140b60 0x7f82c80aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c6ffd700 1 -- 172.21.15.38:0/239293517 learned_addr learned my addr 172.21.15.38:0/239293517 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c77fe700 1 -- 172.21.15.38:0/239293517 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82c8140b60 msgr2=0x7f82c80aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c77fe700 1 --2- 172.21.15.38:0/239293517 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82c8140b60 0x7f82c80aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c77fe700 1 -- 172.21.15.38:0/239293517 >> v2:172.21.15.38:3300/0 conn(0x7f82c8140790 msgr2=0x7f82c80c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c77fe700 1 --2- 172.21.15.38:0/239293517 >> v2:172.21.15.38:3300/0 conn(0x7f82c8140790 0x7f82c80c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c77fe700 1 -- 172.21.15.38:0/239293517 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c80c8c80 con 0x7f82c806e700 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c77fe700 1 --2- 172.21.15.38:0/239293517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82c806e700 0x7f82c80af100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f82c80afb10 tx=0x7f82c0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb17c083bc6505da server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c5ffb700 1 -- 172.21.15.38:0/239293517 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c000e070 con 0x7f82c806e700 2024-08-15T18:01:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c5ffb700 1 -- 172.21.15.38:0/239293517 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c000bb40 con 0x7f82c806e700 2024-08-15T18:01:45.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c5ffb700 1 -- 172.21.15.38:0/239293517 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0012430 con 0x7f82c806e700 2024-08-15T18:01:45.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 -- 172.21.15.38:0/239293517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82c806e700 msgr2=0x7f82c80af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 --2- 172.21.15.38:0/239293517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82c806e700 0x7f82c80af100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f82c80afb10 tx=0x7f82c0009780 comp rx=0 tx=0).stop 2024-08-15T18:01:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 -- 172.21.15.38:0/239293517 shutdown_connections 2024-08-15T18:01:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 --2- 172.21.15.38:0/239293517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82c806e700 0x7f82c80af100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 --2- 172.21.15.38:0/239293517 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82c8140b60 0x7f82c80aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 --2- 172.21.15.38:0/239293517 >> v2:172.21.15.38:3300/0 conn(0x7f82c8140790 0x7f82c80c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.586+0000 7f82c7fff700 1 -- 172.21.15.38:0/239293517 >> 172.21.15.38:0/239293517 conn(0x7f82c80ca2c0 msgr2=0x7f82c806dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c7fff700 1 -- 172.21.15.38:0/239293517 shutdown_connections 2024-08-15T18:01:45.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c7fff700 1 -- 172.21.15.38:0/239293517 wait complete. 2024-08-15T18:01:45.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c7fff700 1 Processor -- start 2024-08-15T18:01:45.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c7fff700 1 -- start start 2024-08-15T18:01:45.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82c806e700 0x7f82c80c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8140790 0x7f82c8069490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82c8140b60 0x7f82c8156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c7fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c80699d0 con 0x7f82c8140790 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c7fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c8156fa0 con 0x7f82c8140b60 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c7fff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c806a4a0 con 0x7f82c806e700 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8140790 0x7f82c8069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8140790 0x7f82c8069490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44148/0 (socket says 172.21.15.38:44148) 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c67fc700 1 -- 172.21.15.38:0/979068808 learned_addr learned my addr 172.21.15.38:0/979068808 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c77fe700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82c8140b60 0x7f82c8156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.590+0000 7f82c6ffd700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82c806e700 0x7f82c80c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82c67fc700 1 -- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82c806e700 msgr2=0x7f82c80c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82c67fc700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82c806e700 0x7f82c80c1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82c67fc700 1 -- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82c8140b60 msgr2=0x7f82c8156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82c67fc700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82c8140b60 0x7f82c8156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82c67fc700 1 -- 172.21.15.38:0/979068808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0009a20 con 0x7f82c8140790 2024-08-15T18:01:45.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82c67fc700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8140790 0x7f82c8069490 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00ec80 tx=0x7f82bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:45.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82aeffd700 1 -- 172.21.15.38:0/979068808 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00c850 con 0x7f82c8140790 2024-08-15T18:01:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82aeffd700 1 -- 172.21.15.38:0/979068808 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc00c9b0 con 0x7f82c8140790 2024-08-15T18:01:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82c7fff700 1 -- 172.21.15.38:0/979068808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c8069c00 con 0x7f82c8140790 2024-08-15T18:01:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82aeffd700 1 -- 172.21.15.38:0/979068808 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00cd50 con 0x7f82c8140790 2024-08-15T18:01:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82c7fff700 1 -- 172.21.15.38:0/979068808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c806a1c0 con 0x7f82c8140790 2024-08-15T18:01:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82c7fff700 1 -- 172.21.15.38:0/979068808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c80c2430 con 0x7f82c8140790 2024-08-15T18:01:45.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82aeffd700 1 -- 172.21.15.38:0/979068808 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f82bc013680 con 0x7f82c8140790 2024-08-15T18:01:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.594+0000 7f82aeffd700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f82b4060860 0x7f82b4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.598+0000 7f82aeffd700 1 -- 172.21.15.38:0/979068808 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f82bc07d1f0 con 0x7f82c8140790 2024-08-15T18:01:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.598+0000 7f82c6ffd700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f82b4060860 0x7f82b4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.598+0000 7f82c6ffd700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f82b4060860 0x7f82b4062d10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f82c80cbbb0 tx=0x7f82b8009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:45.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.606+0000 7f82aeffd700 1 -- 172.21.15.38:0/979068808 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f82bc04d5c0 con 0x7f82c8140790 2024-08-15T18:01:45.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.770+0000 7f82c7fff700 1 -- 172.21.15.38:0/979068808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000845064.meta", "var": "pg_num"} v 0) v1 -- 0x7f82c80638f0 con 0x7f82c8140790 2024-08-15T18:01:45.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82aeffd700 1 -- 172.21.15.38:0/979068808 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000845064.meta", "var": "pg_num"}]=0 v40) v1 ==== 124+0+11 (secure 0 0 0) 0x7f82bc050be0 con 0x7f82c8140790 2024-08-15T18:01:45.776 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T18:01:45.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 -- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f82b4060860 msgr2=0x7f82b4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f82b4060860 0x7f82b4062d10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f82c80cbbb0 tx=0x7f82b8009250 comp rx=0 tx=0).stop 2024-08-15T18:01:45.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 -- 172.21.15.38:0/979068808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8140790 msgr2=0x7f82c8069490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8140790 0x7f82c8069490 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00ec80 tx=0x7f82bc0091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 -- 172.21.15.38:0/979068808 shutdown_connections 2024-08-15T18:01:45.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f82b4060860 0x7f82b4062d10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82c8140b60 0x7f82c8156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8140790 0x7f82c8069490 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 --2- 172.21.15.38:0/979068808 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82c806e700 0x7f82c80c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 -- 172.21.15.38:0/979068808 >> 172.21.15.38:0/979068808 conn(0x7f82c80ca2c0 msgr2=0x7f82c80b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:45.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 -- 172.21.15.38:0/979068808 shutdown_connections 2024-08-15T18:01:45.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.774+0000 7f82c7fff700 1 -- 172.21.15.38:0/979068808 wait complete. 2024-08-15T18:01:45.791 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000845064.data pg_num 2024-08-15T18:01:45.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.898+0000 7f5f16d41700 1 Processor -- start 2024-08-15T18:01:45.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.898+0000 7f5f16d41700 1 -- start start 2024-08-15T18:01:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5f10067510 0x7f5f100ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5f10159480 0x7f5f100cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f100ce210 con 0x7f5f101590b0 2024-08-15T18:01:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f100cba00 con 0x7f5f10159480 2024-08-15T18:01:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f100cdc90 con 0x7f5f10067510 2024-08-15T18:01:45.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16540700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5f10159480 0x7f5f100cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16540700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5f10159480 0x7f5f100cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44162/0 (socket says 172.21.15.38:44162) 2024-08-15T18:01:45.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16540700 1 -- 172.21.15.38:0/1959189801 learned_addr learned my addr 172.21.15.38:0/1959189801 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:45.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f1553e700 1 --2- 172.21.15.38:0/1959189801 >> v2:172.21.15.104:3300/0 conn(0x7f5f10067510 0x7f5f100ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16540700 1 -- 172.21.15.38:0/1959189801 >> v2:172.21.15.104:3300/0 conn(0x7f5f10067510 msgr2=0x7f5f100ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:45.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16540700 1 --2- 172.21.15.38:0/1959189801 >> v2:172.21.15.104:3300/0 conn(0x7f5f10067510 0x7f5f100ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16540700 1 -- 172.21.15.38:0/1959189801 >> v1:172.21.15.104:6789/0 conn(0x7f5f101590b0 legacy=0x7f5f10066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16540700 1 -- 172.21.15.38:0/1959189801 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f100cb440 con 0x7f5f10159480 2024-08-15T18:01:45.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f1553e700 1 --2- 172.21.15.38:0/1959189801 >> v2:172.21.15.104:3300/0 conn(0x7f5f10067510 0x7f5f100ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:45.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16540700 1 --2- 172.21.15.38:0/1959189801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f10159480 0x7f5f100cebe0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5f10067c20 tx=0x7f5f08009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0f15b7127b4a2f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f14d3d700 1 -- 172.21.15.38:0/1959189801 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0800e070 con 0x7f5f10159480 2024-08-15T18:01:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f14d3d700 1 -- 172.21.15.38:0/1959189801 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0800ba50 con 0x7f5f10159480 2024-08-15T18:01:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f14d3d700 1 -- 172.21.15.38:0/1959189801 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080092e0 con 0x7f5f10159480 2024-08-15T18:01:45.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 -- 172.21.15.38:0/1959189801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f10159480 msgr2=0x7f5f100cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 --2- 172.21.15.38:0/1959189801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f10159480 0x7f5f100cebe0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5f10067c20 tx=0x7f5f08009840 comp rx=0 tx=0).stop 2024-08-15T18:01:45.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 -- 172.21.15.38:0/1959189801 shutdown_connections 2024-08-15T18:01:45.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 --2- 172.21.15.38:0/1959189801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f10159480 0x7f5f100cebe0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 --2- 172.21.15.38:0/1959189801 >> v2:172.21.15.104:3300/0 conn(0x7f5f10067510 0x7f5f100ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 -- 172.21.15.38:0/1959189801 >> 172.21.15.38:0/1959189801 conn(0x7f5f100d0b90 msgr2=0x7f5f100ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:45.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 -- 172.21.15.38:0/1959189801 shutdown_connections 2024-08-15T18:01:45.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.902+0000 7f5f16d41700 1 -- 172.21.15.38:0/1959189801 wait complete. 2024-08-15T18:01:45.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f16d41700 1 Processor -- start 2024-08-15T18:01:45.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f16d41700 1 -- start start 2024-08-15T18:01:45.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f16d41700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f10067510 0x7f5f100c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f16d41700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f101590b0 0x7f5f1016c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f16d41700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f10159480 0x7f5f1015d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f16d41700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f10143470 con 0x7f5f10067510 2024-08-15T18:01:45.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f16d41700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1015dd20 con 0x7f5f10159480 2024-08-15T18:01:45.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f16d41700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100c7800 con 0x7f5f101590b0 2024-08-15T18:01:45.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f15d3f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f10067510 0x7f5f100c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f15d3f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f10067510 0x7f5f100c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44176/0 (socket says 172.21.15.38:44176) 2024-08-15T18:01:45.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f15d3f700 1 -- 172.21.15.38:0/675635704 learned_addr learned my addr 172.21.15.38:0/675635704 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:45.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f16540700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f10159480 0x7f5f1015d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f1553e700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f101590b0 0x7f5f1016c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f15d3f700 1 -- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f101590b0 msgr2=0x7f5f1016c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f15d3f700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f101590b0 0x7f5f1016c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f15d3f700 1 -- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f10159480 msgr2=0x7f5f1015d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f15d3f700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f10159480 0x7f5f1015d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f15d3f700 1 -- 172.21.15.38:0/675635704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f08009ae0 con 0x7f5f10067510 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f15d3f700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f10067510 0x7f5f100c7170 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c790 tx=0x7f5f0c007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f05ffb700 1 -- 172.21.15.38:0/675635704 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c011590 con 0x7f5f10067510 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f05ffb700 1 -- 172.21.15.38:0/675635704 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c011710 con 0x7f5f10067510 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f16d41700 1 -- 172.21.15.38:0/675635704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f100c7ae0 con 0x7f5f10067510 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f05ffb700 1 -- 172.21.15.38:0/675635704 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c011ad0 con 0x7f5f10067510 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.906+0000 7f5f16d41700 1 -- 172.21.15.38:0/675635704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f100c80a0 con 0x7f5f10067510 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.910+0000 7f5f16d41700 1 -- 172.21.15.38:0/675635704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f10064330 con 0x7f5f10067510 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.910+0000 7f5f05ffb700 1 -- 172.21.15.38:0/675635704 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5f0c01b3b0 con 0x7f5f10067510 2024-08-15T18:01:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.910+0000 7f5f05ffb700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ef4060860 0x7f5ef4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.910+0000 7f5f05ffb700 1 -- 172.21.15.38:0/675635704 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f5f0c07c130 con 0x7f5f10067510 2024-08-15T18:01:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.910+0000 7f5f1553e700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ef4060860 0x7f5ef4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.910+0000 7f5f1553e700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ef4060860 0x7f5ef4062d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5f00006f00 tx=0x7f5f00008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:45.918+0000 7f5f05ffb700 1 -- 172.21.15.38:0/675635704 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5f0c04c500 con 0x7f5f10067510 2024-08-15T18:01:46.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.082+0000 7f5f16d41700 1 -- 172.21.15.38:0/675635704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000845064.data", "var": "pg_num"} v 0) v1 -- 0x7f5f100d1000 con 0x7f5f10067510 2024-08-15T18:01:46.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.082+0000 7f5f05ffb700 1 -- 172.21.15.38:0/675635704 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000845064.data", "var": "pg_num"}]=0 v40) v1 ==== 124+0+11 (secure 0 0 0) 0x7f5f0c017020 con 0x7f5f10067510 2024-08-15T18:01:46.087 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-08-15T18:01:46.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 -- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ef4060860 msgr2=0x7f5ef4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ef4060860 0x7f5ef4062d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5f00006f00 tx=0x7f5f00008040 comp rx=0 tx=0).stop 2024-08-15T18:01:46.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 -- 172.21.15.38:0/675635704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f10067510 msgr2=0x7f5f100c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f10067510 0x7f5f100c7170 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c790 tx=0x7f5f0c007f40 comp rx=0 tx=0).stop 2024-08-15T18:01:46.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 -- 172.21.15.38:0/675635704 shutdown_connections 2024-08-15T18:01:46.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5ef4060860 0x7f5ef4062d10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f10159480 0x7f5f1015d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f101590b0 0x7f5f1016c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 --2- 172.21.15.38:0/675635704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f10067510 0x7f5f100c7170 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 -- 172.21.15.38:0/675635704 >> 172.21.15.38:0/675635704 conn(0x7f5f100d0b90 msgr2=0x7f5f10066e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:46.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 -- 172.21.15.38:0/675635704 shutdown_connections 2024-08-15T18:01:46.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.086+0000 7f5f16d41700 1 -- 172.21.15.38:0/675635704 wait complete. 2024-08-15T18:01:46.102 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T18:01:46.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631ef4a700 1 Processor -- start 2024-08-15T18:01:46.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631ef4a700 1 -- start start 2024-08-15T18:01:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631ef4a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f63180ab540 0x7f631806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631ef4a700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63180aba70 con 0x7f631806e9d0 2024-08-15T18:01:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631ef4a700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63180abd50 con 0x7f6318155720 2024-08-15T18:01:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631ef4a700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63180ac030 con 0x7f63180ab540 2024-08-15T18:01:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631d747700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f63180ab540 0x7f631806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631df48700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f631806e9d0 0x7f6318155010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50586/0 (socket says 172.21.15.38:50586) 2024-08-15T18:01:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631df48700 1 -- 172.21.15.38:0/655405061 learned_addr learned my addr 172.21.15.38:0/655405061 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:46.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631d747700 1 -- 172.21.15.38:0/655405061 >> v1:172.21.15.104:6789/0 conn(0x7f6318155720 legacy=0x7f63180acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631d747700 1 -- 172.21.15.38:0/655405061 >> v1:172.21.15.38:6789/0 conn(0x7f631806e9d0 legacy=0x7f6318155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631cf46700 1 -- 172.21.15.38:0/655405061 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466114138 0 0) 0x7f63180aba70 con 0x7f631806e9d0 2024-08-15T18:01:46.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631d747700 1 -- 172.21.15.38:0/655405061 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63180ac310 con 0x7f63180ab540 2024-08-15T18:01:46.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631d747700 1 --2- 172.21.15.38:0/655405061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63180ab540 0x7f631806eda0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f63080098e0 tx=0x7f63080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64600f21ab843363 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:46.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631cf46700 1 -- 172.21.15.38:0/655405061 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308009e80 con 0x7f63180ab540 2024-08-15T18:01:46.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631cf46700 1 -- 172.21.15.38:0/655405061 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630800b860 con 0x7f63180ab540 2024-08-15T18:01:46.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.210+0000 7f631cf46700 1 -- 172.21.15.38:0/655405061 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630800bbe0 con 0x7f63180ab540 2024-08-15T18:01:46.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.214+0000 7f631ef4a700 1 -- 172.21.15.38:0/655405061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63180ab540 msgr2=0x7f631806eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.214+0000 7f631ef4a700 1 --2- 172.21.15.38:0/655405061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63180ab540 0x7f631806eda0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f63080098e0 tx=0x7f63080091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.214+0000 7f631ef4a700 1 -- 172.21.15.38:0/655405061 shutdown_connections 2024-08-15T18:01:46.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.214+0000 7f631ef4a700 1 --2- 172.21.15.38:0/655405061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63180ab540 0x7f631806eda0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.214+0000 7f631ef4a700 1 -- 172.21.15.38:0/655405061 >> 172.21.15.38:0/655405061 conn(0x7f63180678d0 msgr2=0x7f6318067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:46.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.214+0000 7f631ef4a700 1 -- 172.21.15.38:0/655405061 shutdown_connections 2024-08-15T18:01:46.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.214+0000 7f631ef4a700 1 -- 172.21.15.38:0/655405061 wait complete. 2024-08-15T18:01:46.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.214+0000 7f631ef4a700 1 Processor -- start 2024-08-15T18:01:46.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.214+0000 7f631ef4a700 1 -- start start 2024-08-15T18:01:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631ef4a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631806e9d0 0x7f631813a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631ef4a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f63180ab540 0x7f6318152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631ef4a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6318155720 0x7f631813d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631ef4a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631813aca0 con 0x7f6318155720 2024-08-15T18:01:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631ef4a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63181646d0 con 0x7f631806e9d0 2024-08-15T18:01:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631ef4a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6318152e10 con 0x7f63180ab540 2024-08-15T18:01:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631e749700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6318155720 0x7f631813d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631d747700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f63180ab540 0x7f6318152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631e749700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6318155720 0x7f631813d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44204/0 (socket says 172.21.15.38:44204) 2024-08-15T18:01:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631e749700 1 -- 172.21.15.38:0/1613895108 learned_addr learned my addr 172.21.15.38:0/1613895108 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631df48700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631806e9d0 0x7f631813a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631e749700 1 -- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f63180ab540 msgr2=0x7f6318152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631e749700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f63180ab540 0x7f6318152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631e749700 1 -- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631806e9d0 msgr2=0x7f631813a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631e749700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631806e9d0 0x7f631813a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631e749700 1 -- 172.21.15.38:0/1613895108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6308009420 con 0x7f6318155720 2024-08-15T18:01:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631e749700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6318155720 0x7f631813d230 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f631000c840 tx=0x7f6310001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f630dffb700 1 -- 172.21.15.38:0/1613895108 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310012630 con 0x7f6318155720 2024-08-15T18:01:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f630dffb700 1 -- 172.21.15.38:0/1613895108 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63100127b0 con 0x7f6318155720 2024-08-15T18:01:46.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631ef4a700 1 -- 172.21.15.38:0/1613895108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63181530f0 con 0x7f6318155720 2024-08-15T18:01:46.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f630dffb700 1 -- 172.21.15.38:0/1613895108 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310012b70 con 0x7f6318155720 2024-08-15T18:01:46.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631ef4a700 1 -- 172.21.15.38:0/1613895108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63181533d0 con 0x7f6318155720 2024-08-15T18:01:46.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.218+0000 7f631ef4a700 1 -- 172.21.15.38:0/1613895108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6318164b90 con 0x7f6318155720 2024-08-15T18:01:46.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.222+0000 7f630dffb700 1 -- 172.21.15.38:0/1613895108 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f631001a3a0 con 0x7f6318155720 2024-08-15T18:01:46.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.222+0000 7f630dffb700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f62fc060860 0x7f62fc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.222+0000 7f630dffb700 1 -- 172.21.15.38:0/1613895108 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f631007cda0 con 0x7f6318155720 2024-08-15T18:01:46.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.222+0000 7f631df48700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f62fc060860 0x7f62fc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.222+0000 7f631df48700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f62fc060860 0x7f62fc062d10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6314007fd0 tx=0x7f6314007d10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:46.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.230+0000 7f630dffb700 1 -- 172.21.15.38:0/1613895108 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f631004d170 con 0x7f6318155720 2024-08-15T18:01:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f631ef4a700 1 -- 172.21.15.38:0/1613895108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63180643a0 con 0x7f6318155720 2024-08-15T18:01:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f630dffb700 1 -- 172.21.15.38:0/1613895108 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4612 (secure 0 0 0) 0x7f6310050790 con 0x7f6318155720 2024-08-15T18:01:46.429 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:46.429 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/373633111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":373633111},{"type":"v1","addr":"172.21.15.38:6833","nonce":373633111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/389886189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":389886189},{"type":"v1","addr":"172.21.15.38:6835","nonce":389886189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3362365103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3362365103},{"type":"v1","addr":"172.21.15.104:6837","nonce":3362365103}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:35.121185+0000","modified":"2024-08-15T17:59:36.892161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4981":{"gid":4981,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6835/875828080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":875828080},{"type":"v1","addr":"172.21.15.104:6835","nonce":875828080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000845064","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T18:01:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f631ef4a700 1 -- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f62fc060860 msgr2=0x7f62fc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f631ef4a700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f62fc060860 0x7f62fc062d10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6314007fd0 tx=0x7f6314007d10 comp rx=0 tx=0).stop 2024-08-15T18:01:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f631ef4a700 1 -- 172.21.15.38:0/1613895108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6318155720 msgr2=0x7f631813d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f631ef4a700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6318155720 0x7f631813d230 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f631000c840 tx=0x7f6310001670 comp rx=0 tx=0).stop 2024-08-15T18:01:46.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f631ef4a700 1 -- 172.21.15.38:0/1613895108 shutdown_connections 2024-08-15T18:01:46.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f631ef4a700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f62fc060860 0x7f62fc062d10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f631ef4a700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6318155720 0x7f631813d230 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f631ef4a700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f63180ab540 0x7f6318152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f631ef4a700 1 --2- 172.21.15.38:0/1613895108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631806e9d0 0x7f631813a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.426+0000 7f631ef4a700 1 -- 172.21.15.38:0/1613895108 >> 172.21.15.38:0/1613895108 conn(0x7f63180678d0 msgr2=0x7f6318063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:46.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.430+0000 7f631ef4a700 1 -- 172.21.15.38:0/1613895108 shutdown_connections 2024-08-15T18:01:46.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.430+0000 7f631ef4a700 1 -- 172.21.15.38:0/1613895108 wait complete. 2024-08-15T18:01:46.433 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-08-15T18:01:46.443 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T18:01:46.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 Processor -- start 2024-08-15T18:01:46.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 -- start start 2024-08-15T18:01:46.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb134156d40 0x7fb134157110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb13406d2f0 0x7fb13406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb13415af40 0x7fb13415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1340addb0 con 0x7fb13415af40 2024-08-15T18:01:46.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1340ae930 con 0x7fb134156d40 2024-08-15T18:01:46.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13406dc80 con 0x7fb13406d2f0 2024-08-15T18:01:46.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb13359e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb13415af40 0x7fb13415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb13359e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb13415af40 0x7fb13415b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44206/0 (socket says 172.21.15.38:44206) 2024-08-15T18:01:46.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb13359e700 1 -- 172.21.15.38:0/935029099 learned_addr learned my addr 172.21.15.38:0/935029099 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:46.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb13259c700 1 --2- 172.21.15.38:0/935029099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb13406d2f0 0x7fb13406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb132d9d700 1 --2- 172.21.15.38:0/935029099 >> v2:172.21.15.104:3300/0 conn(0x7fb134156d40 0x7fb134157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb13359e700 1 -- 172.21.15.38:0/935029099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb13406d2f0 msgr2=0x7fb13406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:46.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb13359e700 1 --2- 172.21.15.38:0/935029099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb13406d2f0 0x7fb13406d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb13359e700 1 -- 172.21.15.38:0/935029099 >> v2:172.21.15.104:3300/0 conn(0x7fb134156d40 msgr2=0x7fb134157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:46.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb13359e700 1 --2- 172.21.15.38:0/935029099 >> v2:172.21.15.104:3300/0 conn(0x7fb134156d40 0x7fb134157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb13359e700 1 -- 172.21.15.38:0/935029099 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340ae000 con 0x7fb13415af40 2024-08-15T18:01:46.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb13259c700 1 --2- 172.21.15.38:0/935029099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb13406d2f0 0x7fb13406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:46.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb132d9d700 1 --2- 172.21.15.38:0/935029099 >> v2:172.21.15.104:3300/0 conn(0x7fb134156d40 0x7fb134157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:46.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb13359e700 1 --2- 172.21.15.38:0/935029099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb13415af40 0x7fb13415b3a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb1340b40a0 tx=0x7fb124009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171e1536138d1cf9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:46.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb131d9b700 1 -- 172.21.15.38:0/935029099 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12400b860 con 0x7fb13415af40 2024-08-15T18:01:46.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb131d9b700 1 -- 172.21.15.38:0/935029099 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12400b9c0 con 0x7fb13415af40 2024-08-15T18:01:46.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb131d9b700 1 -- 172.21.15.38:0/935029099 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12400bd60 con 0x7fb13415af40 2024-08-15T18:01:46.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 -- 172.21.15.38:0/935029099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb13415af40 msgr2=0x7fb13415b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 --2- 172.21.15.38:0/935029099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb13415af40 0x7fb13415b3a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb1340b40a0 tx=0x7fb124009250 comp rx=0 tx=0).stop 2024-08-15T18:01:46.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 -- 172.21.15.38:0/935029099 shutdown_connections 2024-08-15T18:01:46.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 --2- 172.21.15.38:0/935029099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb13415af40 0x7fb13415b3a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 --2- 172.21.15.38:0/935029099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb13406d2f0 0x7fb13406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 --2- 172.21.15.38:0/935029099 >> v2:172.21.15.104:3300/0 conn(0x7fb134156d40 0x7fb134157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 -- 172.21.15.38:0/935029099 >> 172.21.15.38:0/935029099 conn(0x7fb1340ceb30 msgr2=0x7fb13406cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:46.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 -- 172.21.15.38:0/935029099 shutdown_connections 2024-08-15T18:01:46.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.546+0000 7fb139024700 1 -- 172.21.15.38:0/935029099 wait complete. 2024-08-15T18:01:46.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.550+0000 7fb139024700 1 Processor -- start 2024-08-15T18:01:46.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.550+0000 7fb139024700 1 -- start start 2024-08-15T18:01:46.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb139024700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb13406d2f0 0x7fb1341512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb139024700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb134156d40 0x7fb134154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb139024700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb13415af40 0x7fb13413e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb139024700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134154800 con 0x7fb13415af40 2024-08-15T18:01:46.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb139024700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134153bf0 con 0x7fb134156d40 2024-08-15T18:01:46.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb139024700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb134153ed0 con 0x7fb13406d2f0 2024-08-15T18:01:46.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb13359e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb13415af40 0x7fb13413e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb13359e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb13415af40 0x7fb13413e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44220/0 (socket says 172.21.15.38:44220) 2024-08-15T18:01:46.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb13359e700 1 -- 172.21.15.38:0/1096036500 learned_addr learned my addr 172.21.15.38:0/1096036500 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:46.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb13259c700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb134156d40 0x7fb134154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb132d9d700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb13406d2f0 0x7fb1341512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb13359e700 1 -- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb13406d2f0 msgr2=0x7fb1341512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb13359e700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb13406d2f0 0x7fb1341512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb13359e700 1 -- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb134156d40 msgr2=0x7fb134154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb13359e700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb134156d40 0x7fb134154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb13359e700 1 -- 172.21.15.38:0/1096036500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1240094f0 con 0x7fb13415af40 2024-08-15T18:01:46.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb13259c700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb134156d40 0x7fb134154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:01:46.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb13359e700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb13415af40 0x7fb13413e520 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb1240090b0 tx=0x7fb124017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:46.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb12affd700 1 -- 172.21.15.38:0/1096036500 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124011e60 con 0x7fb13415af40 2024-08-15T18:01:46.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb12affd700 1 -- 172.21.15.38:0/1096036500 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12401f070 con 0x7fb13415af40 2024-08-15T18:01:46.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb139024700 1 -- 172.21.15.38:0/1096036500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1341541b0 con 0x7fb13415af40 2024-08-15T18:01:46.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb12affd700 1 -- 172.21.15.38:0/1096036500 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12401a870 con 0x7fb13415af40 2024-08-15T18:01:46.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb139024700 1 -- 172.21.15.38:0/1096036500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1341517f0 con 0x7fb13415af40 2024-08-15T18:01:46.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.554+0000 7fb139024700 1 -- 172.21.15.38:0/1096036500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1340645c0 con 0x7fb13415af40 2024-08-15T18:01:46.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.558+0000 7fb12affd700 1 -- 172.21.15.38:0/1096036500 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb12401ac20 con 0x7fb13415af40 2024-08-15T18:01:46.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.558+0000 7fb12affd700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb114060860 0x7fb114062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.558+0000 7fb12affd700 1 -- 172.21.15.38:0/1096036500 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fb124021070 con 0x7fb13415af40 2024-08-15T18:01:46.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.558+0000 7fb132d9d700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb114060860 0x7fb114062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.558+0000 7fb132d9d700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb114060860 0x7fb114062d10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb11c006fd0 tx=0x7fb11c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:46.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.562+0000 7fb12affd700 1 -- 172.21.15.38:0/1096036500 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb12404ef10 con 0x7fb13415af40 2024-08-15T18:01:46.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.730+0000 7fb139024700 1 -- 172.21.15.38:0/1096036500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1340b0630 con 0x7fb13415af40 2024-08-15T18:01:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.730+0000 7fb12affd700 1 -- 172.21.15.38:0/1096036500 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+15514 (secure 0 0 0) 0x7fb124015020 con 0x7fb13415af40 2024-08-15T18:01:46.734 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:46.734 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":40,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T18:01:44.812744+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000845064.meta","create_time":"2024-08-15T17:59:33.132275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000845064"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000845064.data","create_time":"2024-08-15T17:59:34.143101+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000845064"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:6837/1865762791":"2024-08-16T18:01:41.797233+0000","172.21.15.104:6836/1865762791":"2024-08-16T18:01:41.797233+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T18:01:46.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 -- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb114060860 msgr2=0x7fb114062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb114060860 0x7fb114062d10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb11c006fd0 tx=0x7fb11c008040 comp rx=0 tx=0).stop 2024-08-15T18:01:46.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 -- 172.21.15.38:0/1096036500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb13415af40 msgr2=0x7fb13413e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb13415af40 0x7fb13413e520 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb1240090b0 tx=0x7fb124017000 comp rx=0 tx=0).stop 2024-08-15T18:01:46.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 -- 172.21.15.38:0/1096036500 shutdown_connections 2024-08-15T18:01:46.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fb114060860 0x7fb114062d10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb13415af40 0x7fb13413e520 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb134156d40 0x7fb134154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 --2- 172.21.15.38:0/1096036500 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb13406d2f0 0x7fb1341512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 -- 172.21.15.38:0/1096036500 >> 172.21.15.38:0/1096036500 conn(0x7fb1340ceb30 msgr2=0x7fb1340cc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:46.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 -- 172.21.15.38:0/1096036500 shutdown_connections 2024-08-15T18:01:46.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.734+0000 7fb139024700 1 -- 172.21.15.38:0/1096036500 wait complete. 2024-08-15T18:01:46.756 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000845064 and related pools 2024-08-15T18:01:46.756 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T18:01:46.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.862+0000 7f44eab5e700 1 Processor -- start 2024-08-15T18:01:46.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.862+0000 7f44eab5e700 1 -- start start 2024-08-15T18:01:46.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.862+0000 7f44eab5e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f44e40b14a0 0x7f44e40cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.862+0000 7f44eab5e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e40cbaa0 con 0x7f44e40cb390 2024-08-15T18:01:46.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.862+0000 7f44eab5e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e4157630 con 0x7f44e4154900 2024-08-15T18:01:46.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.862+0000 7f44eab5e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e4068790 con 0x7f44e40b14a0 2024-08-15T18:01:46.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.862+0000 7f44ea35d700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f44e40cb390 0x7f44e4160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40334/0 (socket says 172.21.15.38:40334) 2024-08-15T18:01:46.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.862+0000 7f44ea35d700 1 -- 172.21.15.38:0/4248529702 learned_addr learned my addr 172.21.15.38:0/4248529702 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:46.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.862+0000 7f44e935b700 1 --2- 172.21.15.38:0/4248529702 >> v2:172.21.15.104:3300/0 conn(0x7f44e40b14a0 0x7f44e40cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.862+0000 7f44e8b5a700 1 -- 172.21.15.38:0/4248529702 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3905599163 0 0) 0x7f44e40cbaa0 con 0x7f44e40cb390 2024-08-15T18:01:46.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.862+0000 7f44e8b5a700 1 -- 172.21.15.38:0/4248529702 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44cc003570 con 0x7f44e40cb390 2024-08-15T18:01:46.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44e935b700 1 -- 172.21.15.38:0/4248529702 >> v1:172.21.15.104:6789/0 conn(0x7f44e4154900 legacy=0x7f44e40b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44e935b700 1 -- 172.21.15.38:0/4248529702 >> v1:172.21.15.38:6789/0 conn(0x7f44e40cb390 legacy=0x7f44e4160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44e935b700 1 -- 172.21.15.38:0/4248529702 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4157910 con 0x7f44e40b14a0 2024-08-15T18:01:46.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44e935b700 1 --2- 172.21.15.38:0/4248529702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e40b14a0 0x7f44e40cae50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009e60 tx=0x7f44d8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1866791f7dc176e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:46.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44e8b5a700 1 -- 172.21.15.38:0/4248529702 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d800e070 con 0x7f44e40b14a0 2024-08-15T18:01:46.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44e8b5a700 1 -- 172.21.15.38:0/4248529702 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d800bb60 con 0x7f44e40b14a0 2024-08-15T18:01:46.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44e8b5a700 1 -- 172.21.15.38:0/4248529702 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8012430 con 0x7f44e40b14a0 2024-08-15T18:01:46.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44eab5e700 1 -- 172.21.15.38:0/4248529702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e40b14a0 msgr2=0x7f44e40cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44eab5e700 1 --2- 172.21.15.38:0/4248529702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e40b14a0 0x7f44e40cae50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009e60 tx=0x7f44d8009780 comp rx=0 tx=0).stop 2024-08-15T18:01:46.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44eab5e700 1 -- 172.21.15.38:0/4248529702 shutdown_connections 2024-08-15T18:01:46.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44eab5e700 1 --2- 172.21.15.38:0/4248529702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e40b14a0 0x7f44e40cae50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44eab5e700 1 -- 172.21.15.38:0/4248529702 >> 172.21.15.38:0/4248529702 conn(0x7f44e40678d0 msgr2=0x7f44e4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:46.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44eab5e700 1 -- 172.21.15.38:0/4248529702 shutdown_connections 2024-08-15T18:01:46.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44eab5e700 1 -- 172.21.15.38:0/4248529702 wait complete. 2024-08-15T18:01:46.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44eab5e700 1 Processor -- start 2024-08-15T18:01:46.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.866+0000 7f44eab5e700 1 -- start start 2024-08-15T18:01:46.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44eab5e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e40b14a0 0x7f44e414ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44eab5e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e40cb390 0x7f44e4169980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44eab5e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44e4154900 0x7f44e4155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44eab5e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e4150450 con 0x7f44e40b14a0 2024-08-15T18:01:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44eab5e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e416c560 con 0x7f44e40cb390 2024-08-15T18:01:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44eab5e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e4169ec0 con 0x7f44e4154900 2024-08-15T18:01:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e9b5c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e40b14a0 0x7f44e414ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e9b5c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e40b14a0 0x7f44e414ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48236/0 (socket says 172.21.15.38:48236) 2024-08-15T18:01:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e9b5c700 1 -- 172.21.15.38:0/2293646841 learned_addr learned my addr 172.21.15.38:0/2293646841 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e935b700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e40cb390 0x7f44e4169980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44ea35d700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44e4154900 0x7f44e4155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e9b5c700 1 -- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44e4154900 msgr2=0x7f44e4155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e9b5c700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44e4154900 0x7f44e4155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e9b5c700 1 -- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e40cb390 msgr2=0x7f44e4169980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e9b5c700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e40cb390 0x7f44e4169980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e9b5c700 1 -- 172.21.15.38:0/2293646841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d8009a20 con 0x7f44e40b14a0 2024-08-15T18:01:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e9b5c700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e40b14a0 0x7f44e414ff10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c7c0 tx=0x7f44d4007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e1ffb700 1 -- 172.21.15.38:0/2293646841 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4011590 con 0x7f44e40b14a0 2024-08-15T18:01:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e1ffb700 1 -- 172.21.15.38:0/2293646841 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d4011710 con 0x7f44e40b14a0 2024-08-15T18:01:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44eab5e700 1 -- 172.21.15.38:0/2293646841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e4150830 con 0x7f44e40b14a0 2024-08-15T18:01:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44e1ffb700 1 -- 172.21.15.38:0/2293646841 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4011ad0 con 0x7f44e40b14a0 2024-08-15T18:01:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.870+0000 7f44eab5e700 1 -- 172.21.15.38:0/2293646841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e416c7b0 con 0x7f44e40b14a0 2024-08-15T18:01:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.874+0000 7f44eab5e700 1 -- 172.21.15.38:0/2293646841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e40643a0 con 0x7f44e40b14a0 2024-08-15T18:01:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.874+0000 7f44e1ffb700 1 -- 172.21.15.38:0/2293646841 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f44d401b3b0 con 0x7f44e40b14a0 2024-08-15T18:01:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.874+0000 7f44e1ffb700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44d0060860 0x7f44d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.874+0000 7f44e1ffb700 1 -- 172.21.15.38:0/2293646841 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f44d407c060 con 0x7f44e40b14a0 2024-08-15T18:01:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.874+0000 7f44e935b700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44d0060860 0x7f44d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.874+0000 7f44e935b700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44d0060860 0x7f44d0062d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44e40b18a0 tx=0x7f44d8019000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:46.882+0000 7f44e1ffb700 1 -- 172.21.15.38:0/2293646841 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f44d404c390 con 0x7f44e40b14a0 2024-08-15T18:01:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.082+0000 7f44eab5e700 1 -- 172.21.15.38:0/2293646841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44e406f5f0 con 0x7f44e40b14a0 2024-08-15T18:01:47.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44e1ffb700 1 -- 172.21.15.38:0/2293646841 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4612 (secure 0 0 0) 0x7f44d4017020 con 0x7f44e40b14a0 2024-08-15T18:01:47.087 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:47.087 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/373633111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":373633111},{"type":"v1","addr":"172.21.15.38:6833","nonce":373633111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/389886189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":389886189},{"type":"v1","addr":"172.21.15.38:6835","nonce":389886189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3362365103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3362365103},{"type":"v1","addr":"172.21.15.104:6837","nonce":3362365103}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:35.121185+0000","modified":"2024-08-15T17:59:36.892161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4981":{"gid":4981,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6835/875828080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":875828080},{"type":"v1","addr":"172.21.15.104:6835","nonce":875828080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000845064","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T18:01:47.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 -- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44d0060860 msgr2=0x7f44d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44d0060860 0x7f44d0062d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44e40b18a0 tx=0x7f44d8019000 comp rx=0 tx=0).stop 2024-08-15T18:01:47.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 -- 172.21.15.38:0/2293646841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e40b14a0 msgr2=0x7f44e414ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e40b14a0 0x7f44e414ff10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c7c0 tx=0x7f44d4007f40 comp rx=0 tx=0).stop 2024-08-15T18:01:47.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 -- 172.21.15.38:0/2293646841 shutdown_connections 2024-08-15T18:01:47.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f44d0060860 0x7f44d0062d10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44e4154900 0x7f44e4155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e40cb390 0x7f44e4169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 --2- 172.21.15.38:0/2293646841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e40b14a0 0x7f44e414ff10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 -- 172.21.15.38:0/2293646841 >> 172.21.15.38:0/2293646841 conn(0x7f44e40678d0 msgr2=0x7f44e406d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:47.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 -- 172.21.15.38:0/2293646841 shutdown_connections 2024-08-15T18:01:47.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.086+0000 7f44eab5e700 1 -- 172.21.15.38:0/2293646841 wait complete. 2024-08-15T18:01:47.091 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-08-15T18:01:47.101 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T18:01:47.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff758cd8700 1 Processor -- start 2024-08-15T18:01:47.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff758cd8700 1 -- start start 2024-08-15T18:01:47.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff758cd8700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff754159040 0x7ff7540cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff758cd8700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff75415d5b0 0x7ff7540d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff758cd8700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7540cf190 con 0x7ff7540d0010 2024-08-15T18:01:47.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff758cd8700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7540d09a0 con 0x7ff754159040 2024-08-15T18:01:47.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff758cd8700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7540ce5e0 con 0x7ff75415d5b0 2024-08-15T18:01:47.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff7537fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff754159040 0x7ff7540cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff752ffd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff7540d0010 0x7ff75415cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40340/0 (socket says 172.21.15.38:40340) 2024-08-15T18:01:47.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff7537fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff754159040 0x7ff7540cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48252/0 (socket says 172.21.15.38:48252) 2024-08-15T18:01:47.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff752ffd700 1 -- 172.21.15.38:0/2196277973 learned_addr learned my addr 172.21.15.38:0/2196277973 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:47.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff753fff700 1 --2- 172.21.15.38:0/2196277973 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff75415d5b0 0x7ff7540d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff7537fe700 1 -- 172.21.15.38:0/2196277973 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff75415d5b0 msgr2=0x7ff7540d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:47.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff7537fe700 1 --2- 172.21.15.38:0/2196277973 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff75415d5b0 0x7ff7540d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff7537fe700 1 -- 172.21.15.38:0/2196277973 >> v1:172.21.15.38:6789/0 conn(0x7ff7540d0010 legacy=0x7ff75415cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff7537fe700 1 -- 172.21.15.38:0/2196277973 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7540ce860 con 0x7ff754159040 2024-08-15T18:01:47.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff7537fe700 1 --2- 172.21.15.38:0/2196277973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff754159040 0x7ff7540cf620 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff7440098e0 tx=0x7ff7440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263f3b18e9858ac2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:47.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff7527fc700 1 -- 172.21.15.38:0/2196277973 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74400b640 con 0x7ff754159040 2024-08-15T18:01:47.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff7527fc700 1 -- 172.21.15.38:0/2196277973 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74400b7a0 con 0x7ff754159040 2024-08-15T18:01:47.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.206+0000 7ff7527fc700 1 -- 172.21.15.38:0/2196277973 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74400bb40 con 0x7ff754159040 2024-08-15T18:01:47.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 -- 172.21.15.38:0/2196277973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff754159040 msgr2=0x7ff7540cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 --2- 172.21.15.38:0/2196277973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff754159040 0x7ff7540cf620 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff7440098e0 tx=0x7ff7440091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 -- 172.21.15.38:0/2196277973 shutdown_connections 2024-08-15T18:01:47.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 --2- 172.21.15.38:0/2196277973 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff75415d5b0 0x7ff7540d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 --2- 172.21.15.38:0/2196277973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff754159040 0x7ff7540cf620 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 -- 172.21.15.38:0/2196277973 >> 172.21.15.38:0/2196277973 conn(0x7ff754068250 msgr2=0x7ff754068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:47.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 -- 172.21.15.38:0/2196277973 shutdown_connections 2024-08-15T18:01:47.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 -- 172.21.15.38:0/2196277973 wait complete. 2024-08-15T18:01:47.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 Processor -- start 2024-08-15T18:01:47.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 -- start start 2024-08-15T18:01:47.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7540d0010 0x7ff7540a7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff754159040 0x7ff7540a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff75415d5b0 0x7ff75416a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7540a72e0 con 0x7ff75415d5b0 2024-08-15T18:01:47.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7540a8920 con 0x7ff7540d0010 2024-08-15T18:01:47.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff758cd8700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7540a69b0 con 0x7ff754159040 2024-08-15T18:01:47.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff7537fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7540d0010 0x7ff7540a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff753fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff75415d5b0 0x7ff75416a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff7537fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7540d0010 0x7ff7540a7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:38690/0 (socket says 172.21.15.38:38690) 2024-08-15T18:01:47.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff7537fe700 1 -- 172.21.15.38:0/3951466508 learned_addr learned my addr 172.21.15.38:0/3951466508 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:47.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.210+0000 7ff752ffd700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff754159040 0x7ff7540a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff7537fe700 1 -- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff754159040 msgr2=0x7ff7540a7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff7537fe700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff754159040 0x7ff7540a7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff7537fe700 1 -- 172.21.15.38:0/3951466508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff75415d5b0 msgr2=0x7ff75416a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff7537fe700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff75415d5b0 0x7ff75416a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff7537fe700 1 -- 172.21.15.38:0/3951466508 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff744009420 con 0x7ff7540d0010 2024-08-15T18:01:47.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff7537fe700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7540d0010 0x7ff7540a7770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff754088ef0 tx=0x7ff744017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:47.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff73b7fe700 1 -- 172.21.15.38:0/3951466508 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74401bc40 con 0x7ff7540d0010 2024-08-15T18:01:47.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff758cd8700 1 -- 172.21.15.38:0/3951466508 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7540a6c90 con 0x7ff7540d0010 2024-08-15T18:01:47.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff73b7fe700 1 -- 172.21.15.38:0/3951466508 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74401bdc0 con 0x7ff7540d0010 2024-08-15T18:01:47.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff73b7fe700 1 -- 172.21.15.38:0/3951466508 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744011670 con 0x7ff7540d0010 2024-08-15T18:01:47.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff758cd8700 1 -- 172.21.15.38:0/3951466508 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7540a8480 con 0x7ff7540d0010 2024-08-15T18:01:47.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff758cd8700 1 -- 172.21.15.38:0/3951466508 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7540a7cb0 con 0x7ff7540d0010 2024-08-15T18:01:47.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff73b7fe700 1 -- 172.21.15.38:0/3951466508 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff744011a20 con 0x7ff7540d0010 2024-08-15T18:01:47.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.214+0000 7ff73b7fe700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff740060860 0x7ff740062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.218+0000 7ff73b7fe700 1 -- 172.21.15.38:0/3951466508 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7ff74407e400 con 0x7ff7540d0010 2024-08-15T18:01:47.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.218+0000 7ff752ffd700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff740060860 0x7ff740062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.218+0000 7ff752ffd700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff740060860 0x7ff740062d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff754153100 tx=0x7ff748007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:47.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.226+0000 7ff73b7fe700 1 -- 172.21.15.38:0/3951466508 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff74404e730 con 0x7ff7540d0010 2024-08-15T18:01:47.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.426+0000 7ff758cd8700 1 -- 172.21.15.38:0/3951466508 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7540a8670 con 0x7ff7540d0010 2024-08-15T18:01:47.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.426+0000 7ff73b7fe700 1 -- 172.21.15.38:0/3951466508 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4612 (secure 0 0 0) 0x7ff744051d50 con 0x7ff7540d0010 2024-08-15T18:01:47.430 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:47.430 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/373633111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":373633111},{"type":"v1","addr":"172.21.15.38:6833","nonce":373633111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/389886189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":389886189},{"type":"v1","addr":"172.21.15.38:6835","nonce":389886189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3362365103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3362365103},{"type":"v1","addr":"172.21.15.104:6837","nonce":3362365103}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:59:35.121185+0000","modified":"2024-08-15T17:59:36.892161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4981":{"gid":4981,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6835/875828080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":875828080},{"type":"v1","addr":"172.21.15.104:6835","nonce":875828080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000845064","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T18:01:47.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 -- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff740060860 msgr2=0x7ff740062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff740060860 0x7ff740062d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff754153100 tx=0x7ff748007eb0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 -- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7540d0010 msgr2=0x7ff7540a7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7540d0010 0x7ff7540a7770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff754088ef0 tx=0x7ff744017000 comp rx=0 tx=0).stop 2024-08-15T18:01:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 -- 172.21.15.38:0/3951466508 shutdown_connections 2024-08-15T18:01:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff740060860 0x7ff740062d10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff75415d5b0 0x7ff75416a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff754159040 0x7ff7540a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 --2- 172.21.15.38:0/3951466508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7540d0010 0x7ff7540a7770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 -- 172.21.15.38:0/3951466508 >> 172.21.15.38:0/3951466508 conn(0x7ff754068250 msgr2=0x7ff7540b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 -- 172.21.15.38:0/3951466508 shutdown_connections 2024-08-15T18:01:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.430+0000 7ff758cd8700 1 -- 172.21.15.38:0/3951466508 wait complete. 2024-08-15T18:01:47.434 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-08-15T18:01:47.445 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T18:01:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f063841d700 1 Processor -- start 2024-08-15T18:01:47.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f063841d700 1 -- start start 2024-08-15T18:01:47.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f063841d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0630155380 0x7f06300a9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f063841d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06300a9f60 con 0x7f0630159e40 2024-08-15T18:01:47.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f063841d700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06300b05b0 con 0x7f0630159300 2024-08-15T18:01:47.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f063841d700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063006e4b0 con 0x7f0630155380 2024-08-15T18:01:47.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06369ba700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0630159e40 0x7f0630155750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40346/0 (socket says 172.21.15.38:40346) 2024-08-15T18:01:47.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06369ba700 1 -- 172.21.15.38:0/3819624949 learned_addr learned my addr 172.21.15.38:0/3819624949 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:47.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06361b9700 1 --2- 172.21.15.38:0/3819624949 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0630155380 0x7f06300a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06351b7700 1 -- 172.21.15.38:0/3819624949 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400823913 0 0) 0x7f06300a9f60 con 0x7f0630159e40 2024-08-15T18:01:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06351b7700 1 -- 172.21.15.38:0/3819624949 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0618003570 con 0x7f0630159e40 2024-08-15T18:01:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06351b7700 1 -- 172.21.15.38:0/3819624949 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069353293 0 0) 0x7f06300b05b0 con 0x7f0630159300 2024-08-15T18:01:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06351b7700 1 -- 172.21.15.38:0/3819624949 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06300a9f60 con 0x7f0630159300 2024-08-15T18:01:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06351b7700 1 -- 172.21.15.38:0/3819624949 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2831866689 0 0) 0x7f0618003570 con 0x7f0630159e40 2024-08-15T18:01:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06361b9700 1 -- 172.21.15.38:0/3819624949 >> v1:172.21.15.104:6789/0 conn(0x7f0630159300 legacy=0x7f0630159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06361b9700 1 -- 172.21.15.38:0/3819624949 >> v1:172.21.15.38:6789/0 conn(0x7f0630159e40 legacy=0x7f0630155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06361b9700 1 -- 172.21.15.38:0/3819624949 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063006e6a0 con 0x7f0630155380 2024-08-15T18:01:47.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.546+0000 7f06361b9700 1 --2- 172.21.15.38:0/3819624949 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0630155380 0x7f06300a9730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0624009910 tx=0x7f06240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfffe547924f4303 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:47.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.550+0000 7f06351b7700 1 -- 172.21.15.38:0/3819624949 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628001470 con 0x7f0630155380 2024-08-15T18:01:47.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.550+0000 7f06351b7700 1 -- 172.21.15.38:0/3819624949 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0624011420 con 0x7f0630155380 2024-08-15T18:01:47.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.550+0000 7f06351b7700 1 -- 172.21.15.38:0/3819624949 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06240117a0 con 0x7f0630155380 2024-08-15T18:01:47.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.550+0000 7f063841d700 1 -- 172.21.15.38:0/3819624949 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0630155380 msgr2=0x7f06300a9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.550+0000 7f063841d700 1 --2- 172.21.15.38:0/3819624949 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0630155380 0x7f06300a9730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0624009910 tx=0x7f06240091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.550+0000 7f063841d700 1 -- 172.21.15.38:0/3819624949 shutdown_connections 2024-08-15T18:01:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.550+0000 7f063841d700 1 --2- 172.21.15.38:0/3819624949 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0630155380 0x7f06300a9730 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.550+0000 7f063841d700 1 -- 172.21.15.38:0/3819624949 >> 172.21.15.38:0/3819624949 conn(0x7f0630066e00 msgr2=0x7f0630067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.550+0000 7f063841d700 1 -- 172.21.15.38:0/3819624949 shutdown_connections 2024-08-15T18:01:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.550+0000 7f063841d700 1 -- 172.21.15.38:0/3819624949 wait complete. 2024-08-15T18:01:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f063841d700 1 Processor -- start 2024-08-15T18:01:47.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f063841d700 1 -- start start 2024-08-15T18:01:47.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f063841d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0630155380 0x7f063014a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f063841d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0630159300 0x7f06300990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f063841d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0630159e40 0x7f06301541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f063841d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630158440 con 0x7f0630155380 2024-08-15T18:01:47.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f063841d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630155750 con 0x7f0630159e40 2024-08-15T18:01:47.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f063841d700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0630156570 con 0x7f0630159300 2024-08-15T18:01:47.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06361b9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0630155380 0x7f063014a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06361b9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0630155380 0x7f063014a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48274/0 (socket says 172.21.15.38:48274) 2024-08-15T18:01:47.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06361b9700 1 -- 172.21.15.38:0/2672426907 learned_addr learned my addr 172.21.15.38:0/2672426907 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:47.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06359b8700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0630159300 0x7f06300990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06369ba700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0630159e40 0x7f06301541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06361b9700 1 -- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0630159300 msgr2=0x7f06300990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06361b9700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0630159300 0x7f06300990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06361b9700 1 -- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0630159e40 msgr2=0x7f06301541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06361b9700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0630159e40 0x7f06301541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06361b9700 1 -- 172.21.15.38:0/2672426907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0624009420 con 0x7f0630155380 2024-08-15T18:01:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06369ba700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0630159e40 0x7f06301541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f06361b9700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0630155380 0x7f063014a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0624009860 tx=0x7f0624016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f062e7fc700 1 -- 172.21.15.38:0/2672426907 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062401cba0 con 0x7f0630155380 2024-08-15T18:01:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f062e7fc700 1 -- 172.21.15.38:0/2672426907 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062401cd20 con 0x7f0630155380 2024-08-15T18:01:47.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f063841d700 1 -- 172.21.15.38:0/2672426907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0630155960 con 0x7f0630155380 2024-08-15T18:01:47.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f062e7fc700 1 -- 172.21.15.38:0/2672426907 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062401a6a0 con 0x7f0630155380 2024-08-15T18:01:47.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.554+0000 7f063841d700 1 -- 172.21.15.38:0/2672426907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0630155f20 con 0x7f0630155380 2024-08-15T18:01:47.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.558+0000 7f063841d700 1 -- 172.21.15.38:0/2672426907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06300645c0 con 0x7f0630155380 2024-08-15T18:01:47.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.558+0000 7f062e7fc700 1 -- 172.21.15.38:0/2672426907 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f062401aad0 con 0x7f0630155380 2024-08-15T18:01:47.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.558+0000 7f062e7fc700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f061c060860 0x7f061c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.558+0000 7f062e7fc700 1 -- 172.21.15.38:0/2672426907 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f062407e3f0 con 0x7f0630155380 2024-08-15T18:01:47.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.558+0000 7f06359b8700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f061c060860 0x7f061c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.562+0000 7f06359b8700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f061c060860 0x7f061c062d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f06301520e0 tx=0x7f0620007f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.566+0000 7f062e7fc700 1 -- 172.21.15.38:0/2672426907 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f062404e7c0 con 0x7f0630155380 2024-08-15T18:01:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.730+0000 7f063841d700 1 -- 172.21.15.38:0/2672426907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0630151c60 con 0x7f0630155380 2024-08-15T18:01:47.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f062e7fc700 1 -- 172.21.15.38:0/2672426907 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+15514 (secure 0 0 0) 0x7f0624051de0 con 0x7f0630155380 2024-08-15T18:01:47.736 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:47.736 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":40,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T18:01:44.812744+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000845064.meta","create_time":"2024-08-15T17:59:33.132275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000845064"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000845064.data","create_time":"2024-08-15T17:59:34.143101+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000845064"}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:6837/1865762791":"2024-08-16T18:01:41.797233+0000","172.21.15.104:6836/1865762791":"2024-08-16T18:01:41.797233+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T18:01:47.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 -- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f061c060860 msgr2=0x7f061c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f061c060860 0x7f061c062d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f06301520e0 tx=0x7f0620007f90 comp rx=0 tx=0).stop 2024-08-15T18:01:47.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 -- 172.21.15.38:0/2672426907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0630155380 msgr2=0x7f063014a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0630155380 0x7f063014a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0624009860 tx=0x7f0624016000 comp rx=0 tx=0).stop 2024-08-15T18:01:47.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 -- 172.21.15.38:0/2672426907 shutdown_connections 2024-08-15T18:01:47.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f061c060860 0x7f061c062d10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0630159e40 0x7f06301541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0630159300 0x7f06300990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 --2- 172.21.15.38:0/2672426907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0630155380 0x7f063014a7a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 -- 172.21.15.38:0/2672426907 >> 172.21.15.38:0/2672426907 conn(0x7f0630066e00 msgr2=0x7f0630152200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:47.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 -- 172.21.15.38:0/2672426907 shutdown_connections 2024-08-15T18:01:47.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.734+0000 7f063841d700 1 -- 172.21.15.38:0/2672426907 wait complete. 2024-08-15T18:01:47.752 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000845064 2024-08-15T18:01:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.862+0000 7f9c7529a700 1 Processor -- start 2024-08-15T18:01:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.862+0000 7f9c7529a700 1 -- start start 2024-08-15T18:01:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.862+0000 7f9c7529a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9c700afc00 0x7f9c70068960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.862+0000 7f9c7529a700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c70069c70 con 0x7f9c70142970 2024-08-15T18:01:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.862+0000 7f9c7529a700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c70069060 con 0x7f9c700b1450 2024-08-15T18:01:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.862+0000 7f9c7529a700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c70069340 con 0x7f9c700afc00 2024-08-15T18:01:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.862+0000 7f9c74a99700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9c700afc00 0x7f9c70068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.862+0000 7f9c6ffff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9c70142970 0x7f9c700b0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40358/0 (socket says 172.21.15.38:40358) 2024-08-15T18:01:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.862+0000 7f9c6ffff700 1 -- 172.21.15.38:0/1197566844 learned_addr learned my addr 172.21.15.38:0/1197566844 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:47.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c74a99700 1 -- 172.21.15.38:0/1197566844 >> v1:172.21.15.104:6789/0 conn(0x7f9c700b1450 legacy=0x7f9c70160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c74a99700 1 -- 172.21.15.38:0/1197566844 >> v1:172.21.15.38:6789/0 conn(0x7f9c70142970 legacy=0x7f9c700b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c6effd700 1 -- 172.21.15.38:0/1197566844 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616918233 0 0) 0x7f9c70069060 con 0x7f9c700b1450 2024-08-15T18:01:47.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c74a99700 1 -- 172.21.15.38:0/1197566844 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c70069620 con 0x7f9c700afc00 2024-08-15T18:01:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c74a99700 1 --2- 172.21.15.38:0/1197566844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c700afc00 0x7f9c70068960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009e60 tx=0x7f9c68009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7f24310a48ee7e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c6effd700 1 -- 172.21.15.38:0/1197566844 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800e070 con 0x7f9c700afc00 2024-08-15T18:01:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c6effd700 1 -- 172.21.15.38:0/1197566844 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6800bb30 con 0x7f9c700afc00 2024-08-15T18:01:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c6effd700 1 -- 172.21.15.38:0/1197566844 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68012430 con 0x7f9c700afc00 2024-08-15T18:01:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c7529a700 1 -- 172.21.15.38:0/1197566844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c700afc00 msgr2=0x7f9c70068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c7529a700 1 --2- 172.21.15.38:0/1197566844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c700afc00 0x7f9c70068960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009e60 tx=0x7f9c68009780 comp rx=0 tx=0).stop 2024-08-15T18:01:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c7529a700 1 -- 172.21.15.38:0/1197566844 shutdown_connections 2024-08-15T18:01:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c7529a700 1 --2- 172.21.15.38:0/1197566844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c700afc00 0x7f9c70068960 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c7529a700 1 -- 172.21.15.38:0/1197566844 >> 172.21.15.38:0/1197566844 conn(0x7f9c700cdeb0 msgr2=0x7f9c700ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c7529a700 1 -- 172.21.15.38:0/1197566844 shutdown_connections 2024-08-15T18:01:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c7529a700 1 -- 172.21.15.38:0/1197566844 wait complete. 2024-08-15T18:01:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.866+0000 7f9c7529a700 1 Processor -- start 2024-08-15T18:01:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c7529a700 1 -- start start 2024-08-15T18:01:47.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c7529a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c700afc00 0x7f9c700cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c7529a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c700b1450 0x7f9c7012d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c7529a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c70142970 0x7f9c700ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c7529a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c700ca4b0 con 0x7f9c700b1450 2024-08-15T18:01:47.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c7529a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c700c98a0 con 0x7f9c70142970 2024-08-15T18:01:47.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c7529a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c700c9b80 con 0x7f9c700afc00 2024-08-15T18:01:47.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c700b1450 0x7f9c7012d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c700b1450 0x7f9c7012d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48298/0 (socket says 172.21.15.38:48298) 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c6f7fe700 1 -- 172.21.15.38:0/3951306102 learned_addr learned my addr 172.21.15.38:0/3951306102 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c74a99700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c70142970 0x7f9c700ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c6ffff700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c700afc00 0x7f9c700cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c6f7fe700 1 -- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c700afc00 msgr2=0x7f9c700cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c6f7fe700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c700afc00 0x7f9c700cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c6f7fe700 1 -- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c70142970 msgr2=0x7f9c700ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c6f7fe700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c70142970 0x7f9c700ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c6f7fe700 1 -- 172.21.15.38:0/3951306102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c68009a20 con 0x7f9c700b1450 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c6f7fe700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c700b1450 0x7f9c7012d930 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c7e0 tx=0x7f9c60007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c67fff700 1 -- 172.21.15.38:0/3951306102 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c600105c0 con 0x7f9c700b1450 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c67fff700 1 -- 172.21.15.38:0/3951306102 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c60010740 con 0x7f9c700b1450 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c7529a700 1 -- 172.21.15.38:0/3951306102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c700c9e60 con 0x7f9c700b1450 2024-08-15T18:01:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c67fff700 1 -- 172.21.15.38:0/3951306102 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60010b00 con 0x7f9c700b1450 2024-08-15T18:01:47.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.870+0000 7f9c7529a700 1 -- 172.21.15.38:0/3951306102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c700cbe70 con 0x7f9c700b1450 2024-08-15T18:01:47.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.874+0000 7f9c7529a700 1 -- 172.21.15.38:0/3951306102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c700638f0 con 0x7f9c700b1450 2024-08-15T18:01:47.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.874+0000 7f9c67fff700 1 -- 172.21.15.38:0/3951306102 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9c6001b3b0 con 0x7f9c700b1450 2024-08-15T18:01:47.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.874+0000 7f9c67fff700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c50060860 0x7f9c50062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:47.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.874+0000 7f9c67fff700 1 -- 172.21.15.38:0/3951306102 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f9c6007c290 con 0x7f9c700b1450 2024-08-15T18:01:47.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.874+0000 7f9c6ffff700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c50060860 0x7f9c50062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:47.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.874+0000 7f9c6ffff700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c50060860 0x7f9c50062d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c700c86d0 tx=0x7f9c58009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:47.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:47.882+0000 7f9c67fff700 1 -- 172.21.15.38:0/3951306102 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9c6004c5c0 con 0x7f9c700b1450 2024-08-15T18:01:48.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.082+0000 7f9c7529a700 1 -- 172.21.15.38:0/3951306102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000845064"} v 0) v1 -- 0x7f9c70067d10 con 0x7f9c700b1450 2024-08-15T18:01:48.087 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-08-15T18:01:48.086+0000 7fa9f6684700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-15T18:01:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.098+0000 7f9c67fff700 1 -- 172.21.15.38:0/3951306102 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000845064"}]=0 volume_0000000000845064 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) v1 ==== 195+0+0 (secure 0 0 0) 0x7f9c6004fbe0 con 0x7f9c700b1450 2024-08-15T18:01:48.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 -- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c50060860 msgr2=0x7f9c50062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:48.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c50060860 0x7f9c50062d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c700c86d0 tx=0x7f9c58009040 comp rx=0 tx=0).stop 2024-08-15T18:01:48.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 -- 172.21.15.38:0/3951306102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c700b1450 msgr2=0x7f9c7012d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:48.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c700b1450 0x7f9c7012d930 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c7e0 tx=0x7f9c60007df0 comp rx=0 tx=0).stop 2024-08-15T18:01:48.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 -- 172.21.15.38:0/3951306102 shutdown_connections 2024-08-15T18:01:48.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f9c50060860 0x7f9c50062d10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:48.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c70142970 0x7f9c700ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:48.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c700b1450 0x7f9c7012d930 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:48.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 --2- 172.21.15.38:0/3951306102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c700afc00 0x7f9c700cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:48.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 -- 172.21.15.38:0/3951306102 >> 172.21.15.38:0/3951306102 conn(0x7f9c700cdeb0 msgr2=0x7f9c701417c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:48.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 -- 172.21.15.38:0/3951306102 shutdown_connections 2024-08-15T18:01:48.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.102+0000 7f9c7529a700 1 -- 172.21.15.38:0/3951306102 wait complete. 2024-08-15T18:01:48.107 INFO:teuthology.orchestra.run.smithi038.stderr:volume_0000000000845064 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-15T18:01:48.121 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000845064 --yes-i-really-mean-it 2024-08-15T18:01:48.135 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-08-15T18:01:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.226+0000 7f1ffcbaa700 1 Processor -- start 2024-08-15T18:01:48.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.226+0000 7f1ffcbaa700 1 -- start start 2024-08-15T18:01:48.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ff80a9040 0x7f1ff80a9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1ff80acf00 0x7f1ff806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff80aa890 con 0x7f1ff80adf20 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff80aa030 con 0x7f1ff80acf00 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80ad330 con 0x7f1ff80a9040 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff6d9d700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1ff80adf20 0x7f1ff81569b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40372/0 (socket says 172.21.15.38:40372) 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff6d9d700 1 -- 172.21.15.38:0/2943712051 learned_addr learned my addr 172.21.15.38:0/2943712051 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff5d9b700 1 --2- 172.21.15.38:0/2943712051 >> v2:172.21.15.104:3300/0 conn(0x7f1ff80acf00 0x7f1ff806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff659c700 1 --2- 172.21.15.38:0/2943712051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ff80a9040 0x7f1ff80a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff559a700 1 -- 172.21.15.38:0/2943712051 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503897365 0 0) 0x7f1ff80aa890 con 0x7f1ff80adf20 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff559a700 1 -- 172.21.15.38:0/2943712051 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe4003570 con 0x7f1ff80adf20 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff5d9b700 1 -- 172.21.15.38:0/2943712051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ff80a9040 msgr2=0x7f1ff80a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff5d9b700 1 --2- 172.21.15.38:0/2943712051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ff80a9040 0x7f1ff80a9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:48.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff5d9b700 1 -- 172.21.15.38:0/2943712051 >> v1:172.21.15.38:6789/0 conn(0x7f1ff80adf20 legacy=0x7f1ff81569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:48.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff5d9b700 1 -- 172.21.15.38:0/2943712051 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff80ae730 con 0x7f1ff80acf00 2024-08-15T18:01:48.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff659c700 1 --2- 172.21.15.38:0/2943712051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ff80a9040 0x7f1ff80a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:01:48.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff5d9b700 1 --2- 172.21.15.38:0/2943712051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ff80acf00 0x7f1ff806d2a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009e60 tx=0x7f1fe8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a3de5564240f1ab server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:48.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff559a700 1 -- 172.21.15.38:0/2943712051 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe800e070 con 0x7f1ff80acf00 2024-08-15T18:01:48.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff559a700 1 -- 172.21.15.38:0/2943712051 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe800bab0 con 0x7f1ff80acf00 2024-08-15T18:01:48.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ff559a700 1 -- 172.21.15.38:0/2943712051 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe800be70 con 0x7f1ff80acf00 2024-08-15T18:01:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/2943712051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ff80acf00 msgr2=0x7f1ff806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 --2- 172.21.15.38:0/2943712051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ff80acf00 0x7f1ff806d2a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009e60 tx=0x7f1fe8009780 comp rx=0 tx=0).stop 2024-08-15T18:01:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/2943712051 shutdown_connections 2024-08-15T18:01:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 --2- 172.21.15.38:0/2943712051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ff80acf00 0x7f1ff806d2a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 --2- 172.21.15.38:0/2943712051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ff80a9040 0x7f1ff80a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/2943712051 >> 172.21.15.38:0/2943712051 conn(0x7f1ff8066e00 msgr2=0x7f1ff8067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/2943712051 shutdown_connections 2024-08-15T18:01:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.230+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/2943712051 wait complete. 2024-08-15T18:01:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ffcbaa700 1 Processor -- start 2024-08-15T18:01:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ffcbaa700 1 -- start start 2024-08-15T18:01:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ffcbaa700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ff80a9040 0x7f1ff81553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ffcbaa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff80acf00 0x7f1ff80cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ffcbaa700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ff80adf20 0x7f1ff80b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ffcbaa700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81690b0 con 0x7f1ff80acf00 2024-08-15T18:01:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ffcbaa700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80b08c0 con 0x7f1ff80a9040 2024-08-15T18:01:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ffcbaa700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff816a040 con 0x7f1ff80adf20 2024-08-15T18:01:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ff5d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff80acf00 0x7f1ff80cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ff5d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff80acf00 0x7f1ff80cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48302/0 (socket says 172.21.15.38:48302) 2024-08-15T18:01:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ff5d9b700 1 -- 172.21.15.38:0/1504128703 learned_addr learned my addr 172.21.15.38:0/1504128703 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ff659c700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ff80a9040 0x7f1ff81553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ff6d9d700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ff80adf20 0x7f1ff80b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ff5d9b700 1 -- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ff80adf20 msgr2=0x7f1ff80b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ff5d9b700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ff80adf20 0x7f1ff80b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ff5d9b700 1 -- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ff80a9040 msgr2=0x7f1ff81553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ff5d9b700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ff80a9040 0x7f1ff81553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ff5d9b700 1 -- 172.21.15.38:0/1504128703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe0009420 con 0x7f1ff80acf00 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.234+0000 7f1ff5d9b700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff80acf00 0x7f1ff80cb5c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80710e0 tx=0x7f1fe8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.238+0000 7f1fee7fc700 1 -- 172.21.15.38:0/1504128703 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8012d80 con 0x7f1ff80acf00 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.238+0000 7f1fee7fc700 1 -- 172.21.15.38:0/1504128703 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe8027470 con 0x7f1ff80acf00 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.238+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/1504128703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe8009a20 con 0x7f1ff80acf00 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.238+0000 7f1fee7fc700 1 -- 172.21.15.38:0/1504128703 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8027830 con 0x7f1ff80acf00 2024-08-15T18:01:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.238+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/1504128703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff8169560 con 0x7f1ff80acf00 2024-08-15T18:01:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.238+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/1504128703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff80645c0 con 0x7f1ff80acf00 2024-08-15T18:01:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.238+0000 7f1fee7fc700 1 -- 172.21.15.38:0/1504128703 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1fe8024070 con 0x7f1ff80acf00 2024-08-15T18:01:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.238+0000 7f1fee7fc700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1fd8060930 0x7f1fd8062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:48.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.238+0000 7f1fee7fc700 1 -- 172.21.15.38:0/1504128703 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5923+0+0 (secure 0 0 0) 0x7f1fe807f090 con 0x7f1ff80acf00 2024-08-15T18:01:48.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.238+0000 7f1ff659c700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1fd8060930 0x7f1fd8062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:48.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.242+0000 7f1ff659c700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1fd8060930 0x7f1fd8062de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80cb150 tx=0x7f1fe00091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.246+0000 7f1fee7fc700 1 -- 172.21.15.38:0/1504128703 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1fe804f480 con 0x7f1ff80acf00 2024-08-15T18:01:48.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:48.450+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/1504128703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000845064", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1ff806efa0 con 0x7f1ff80acf00 2024-08-15T18:01:49.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.106+0000 7f1fee7fc700 1 -- 172.21.15.38:0/1504128703 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000845064", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 121+0+0 (secure 0 0 0) 0x7f1fe8052aa0 con 0x7f1ff80acf00 2024-08-15T18:01:49.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1fd8060930 msgr2=0x7f1fd8062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:49.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1fd8060930 0x7f1fd8062de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80cb150 tx=0x7f1fe00091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:49.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/1504128703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff80acf00 msgr2=0x7f1ff80cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:49.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff80acf00 0x7f1ff80cb5c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80710e0 tx=0x7f1fe8019000 comp rx=0 tx=0).stop 2024-08-15T18:01:49.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/1504128703 shutdown_connections 2024-08-15T18:01:49.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1fd8060930 0x7f1fd8062de0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:49.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ff80adf20 0x7f1ff80b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:49.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff80acf00 0x7f1ff80cb5c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:49.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 --2- 172.21.15.38:0/1504128703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ff80a9040 0x7f1ff81553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/1504128703 >> 172.21.15.38:0/1504128703 conn(0x7f1ff8066e00 msgr2=0x7f1ff8156780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/1504128703 shutdown_connections 2024-08-15T18:01:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.110+0000 7f1ffcbaa700 1 -- 172.21.15.38:0/1504128703 wait complete. 2024-08-15T18:01:49.131 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000845064.meta cephfs.volume_0000000000845064.meta --yes-i-really-really-mean-it 2024-08-15T18:01:49.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.250+0000 7fba7080d700 1 Processor -- start 2024-08-15T18:01:49.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.250+0000 7fba7080d700 1 -- start start 2024-08-15T18:01:49.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba7080d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fba6c0b26d0 0x7fba6c0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:49.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba7080d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c5fd0 0x7fba6c0c6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:49.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba7080d700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba6c0c5b40 con 0x7fba6c0c6960 2024-08-15T18:01:49.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba7080d700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6c0b2aa0 con 0x7fba6c0b26d0 2024-08-15T18:01:49.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba7080d700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c159820 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6affd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c5fd0 0x7fba6c0c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6affd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c5fd0 0x7fba6c0c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.38:41656/0 (socket says 172.21.15.38:41656) 2024-08-15T18:01:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6b7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fba6c0b26d0 0x7fba6c0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6affd700 1 -- 172.21.15.38:0/684298267 learned_addr learned my addr 172.21.15.38:0/684298267 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6bfff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fba6c0c6960 0x7fba6c0cacd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:46646/0 (socket says 172.21.15.38:46646) 2024-08-15T18:01:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6a7fc700 1 -- 172.21.15.38:0/684298267 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097892770 0 0) 0x7fba6c0c5b40 con 0x7fba6c0c6960 2024-08-15T18:01:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6a7fc700 1 -- 172.21.15.38:0/684298267 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba58003570 con 0x7fba6c0c6960 2024-08-15T18:01:49.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6affd700 1 -- 172.21.15.38:0/684298267 >> v2:172.21.15.104:3300/0 conn(0x7fba6c0b26d0 msgr2=0x7fba6c0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:49.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6affd700 1 --2- 172.21.15.38:0/684298267 >> v2:172.21.15.104:3300/0 conn(0x7fba6c0b26d0 0x7fba6c0b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:49.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6affd700 1 -- 172.21.15.38:0/684298267 >> v1:172.21.15.104:6789/0 conn(0x7fba6c0c6960 legacy=0x7fba6c0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:49.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6affd700 1 -- 172.21.15.38:0/684298267 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c0c4f30 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6b7fe700 1 --2- 172.21.15.38:0/684298267 >> v2:172.21.15.104:3300/0 conn(0x7fba6c0b26d0 0x7fba6c0b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:01:49.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6affd700 1 --2- 172.21.15.38:0/684298267 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c5fd0 0x7fba6c0c6420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009990 tx=0x7fba5c009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1646e95aba13add server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:49.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6a7fc700 1 -- 172.21.15.38:0/684298267 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c005c10 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6a7fc700 1 -- 172.21.15.38:0/684298267 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c005d70 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.254+0000 7fba6a7fc700 1 -- 172.21.15.38:0/684298267 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c006240 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 -- 172.21.15.38:0/684298267 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c5fd0 msgr2=0x7fba6c0c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 --2- 172.21.15.38:0/684298267 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c5fd0 0x7fba6c0c6420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009990 tx=0x7fba5c009220 comp rx=0 tx=0).stop 2024-08-15T18:01:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 -- 172.21.15.38:0/684298267 shutdown_connections 2024-08-15T18:01:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 --2- 172.21.15.38:0/684298267 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c5fd0 0x7fba6c0c6420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 --2- 172.21.15.38:0/684298267 >> v2:172.21.15.104:3300/0 conn(0x7fba6c0b26d0 0x7fba6c0b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 -- 172.21.15.38:0/684298267 >> 172.21.15.38:0/684298267 conn(0x7fba6c067590 msgr2=0x7fba6c0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:49.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 -- 172.21.15.38:0/684298267 shutdown_connections 2024-08-15T18:01:49.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 -- 172.21.15.38:0/684298267 wait complete. 2024-08-15T18:01:49.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 Processor -- start 2024-08-15T18:01:49.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 -- start start 2024-08-15T18:01:49.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba6c0b26d0 0x7fba6c156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:49.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba6c0c5fd0 0x7fba6c09cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:49.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c6960 0x7fba6c09d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:49.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c09ddc0 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c09a050 con 0x7fba6c0b26d0 2024-08-15T18:01:49.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba7080d700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c09d800 con 0x7fba6c0c6960 2024-08-15T18:01:49.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba6affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba6c0c5fd0 0x7fba6c09cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:49.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba6affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba6c0c5fd0 0x7fba6c09cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48318/0 (socket says 172.21.15.38:48318) 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba6affd700 1 -- 172.21.15.38:0/1032123218 learned_addr learned my addr 172.21.15.38:0/1032123218 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba6bfff700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c6960 0x7fba6c09d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.258+0000 7fba6b7fe700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba6c0b26d0 0x7fba6c156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba6affd700 1 -- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c6960 msgr2=0x7fba6c09d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba6affd700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c6960 0x7fba6c09d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba6affd700 1 -- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba6c0b26d0 msgr2=0x7fba6c156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba6affd700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba6c0b26d0 0x7fba6c156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba6affd700 1 -- 172.21.15.38:0/1032123218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba54009420 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba6affd700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba6c0c5fd0 0x7fba6c09cc60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fba6c159960 tx=0x7fba5c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba637fe700 1 -- 172.21.15.38:0/1032123218 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c011950 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba637fe700 1 -- 172.21.15.38:0/1032123218 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c011ab0 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba7080d700 1 -- 172.21.15.38:0/1032123218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5c009480 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba637fe700 1 -- 172.21.15.38:0/1032123218 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c011e70 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba7080d700 1 -- 172.21.15.38:0/1032123218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6c0c8920 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba7080d700 1 -- 172.21.15.38:0/1032123218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba6c0644b0 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.262+0000 7fba637fe700 1 -- 172.21.15.38:0/1032123218 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fba5c01b640 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.266+0000 7fba637fe700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba4c060860 0x7fba4c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.266+0000 7fba637fe700 1 -- 172.21.15.38:0/1032123218 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5923+0+0 (secure 0 0 0) 0x7fba5c07e910 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.266+0000 7fba6b7fe700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba4c060860 0x7fba4c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.266+0000 7fba6b7fe700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba4c060860 0x7fba4c062d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0c68e0 tx=0x7fba540091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:49.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.270+0000 7fba637fe700 1 -- 172.21.15.38:0/1032123218 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fba5c04ebe0 con 0x7fba6c0c5fd0 2024-08-15T18:01:49.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:49.438+0000 7fba7080d700 1 -- 172.21.15.38:0/1032123218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000845064.meta", "pool2": "cephfs.volume_0000000000845064.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fba6c0b2280 con 0x7fba6c0c5fd0 2024-08-15T18:01:50.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.114+0000 7fba637fe700 1 -- 172.21.15.38:0/1032123218 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000845064.meta", "pool2": "cephfs.volume_0000000000845064.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000845064.meta' removed v42) v1 ==== 241+0+0 (secure 0 0 0) 0x7fba5c02a020 con 0x7fba6c0c5fd0 2024-08-15T18:01:50.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 -- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba4c060860 msgr2=0x7fba4c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:50.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba4c060860 0x7fba4c062d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0c68e0 tx=0x7fba540091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:50.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 -- 172.21.15.38:0/1032123218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba6c0c5fd0 msgr2=0x7fba6c09cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:50.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba6c0c5fd0 0x7fba6c09cc60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fba6c159960 tx=0x7fba5c016000 comp rx=0 tx=0).stop 2024-08-15T18:01:50.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 -- 172.21.15.38:0/1032123218 shutdown_connections 2024-08-15T18:01:50.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba4c060860 0x7fba4c062d10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:50.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba6c0c6960 0x7fba6c09d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:50.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba6c0c5fd0 0x7fba6c09cc60 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fba6c159960 tx=0x7fba5c016000 comp rx=0 tx=0).stop 2024-08-15T18:01:50.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 --2- 172.21.15.38:0/1032123218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba6c0b26d0 0x7fba6c156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:50.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 -- 172.21.15.38:0/1032123218 >> 172.21.15.38:0/1032123218 conn(0x7fba6c067590 msgr2=0x7fba6c0cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:50.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 -- 172.21.15.38:0/1032123218 shutdown_connections 2024-08-15T18:01:50.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.122+0000 7fba7080d700 1 -- 172.21.15.38:0/1032123218 wait complete. 2024-08-15T18:01:50.126 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs.volume_0000000000845064.meta' removed 2024-08-15T18:01:50.136 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000845064.data cephfs.volume_0000000000845064.data --yes-i-really-really-mean-it 2024-08-15T18:01:50.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.238+0000 7f5f55e12700 1 Processor -- start 2024-08-15T18:01:50.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.238+0000 7f5f55e12700 1 -- start start 2024-08-15T18:01:50.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f55e12700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f500cc560 0x7f5f50160f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:50.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f55e12700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f500cb0c0 con 0x7f5f5015a160 2024-08-15T18:01:50.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f55e12700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f500ca4b0 con 0x7f5f50159d90 2024-08-15T18:01:50.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f55e12700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f500ca790 con 0x7f5f500cc560 2024-08-15T18:01:50.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4effd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5f5015a160 0x7f5f501559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40384/0 (socket says 172.21.15.38:40384) 2024-08-15T18:01:50.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4effd700 1 -- 172.21.15.38:0/2418982990 learned_addr learned my addr 172.21.15.38:0/2418982990 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:50.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4ffff700 1 --2- 172.21.15.38:0/2418982990 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f500cc560 0x7f5f50160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:50.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4e7fc700 1 -- 172.21.15.38:0/2418982990 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996680544 0 0) 0x7f5f500ca4b0 con 0x7f5f50159d90 2024-08-15T18:01:50.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4e7fc700 1 -- 172.21.15.38:0/2418982990 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f38003570 con 0x7f5f50159d90 2024-08-15T18:01:50.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4e7fc700 1 -- 172.21.15.38:0/2418982990 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803491221 0 0) 0x7f5f500cb0c0 con 0x7f5f5015a160 2024-08-15T18:01:50.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4ffff700 1 -- 172.21.15.38:0/2418982990 >> v1:172.21.15.104:6789/0 conn(0x7f5f50159d90 legacy=0x7f5f500c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:50.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4ffff700 1 -- 172.21.15.38:0/2418982990 >> v1:172.21.15.38:6789/0 conn(0x7f5f5015a160 legacy=0x7f5f501559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:50.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4ffff700 1 -- 172.21.15.38:0/2418982990 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f500caa70 con 0x7f5f500cc560 2024-08-15T18:01:50.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4ffff700 1 --2- 172.21.15.38:0/2418982990 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f500cc560 0x7f5f50160f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009e90 tx=0x7f5f48009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dcb61a92fc457b6 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:50.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4e7fc700 1 -- 172.21.15.38:0/2418982990 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4800e070 con 0x7f5f500cc560 2024-08-15T18:01:50.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4e7fc700 1 -- 172.21.15.38:0/2418982990 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4800bc30 con 0x7f5f500cc560 2024-08-15T18:01:50.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.242+0000 7f5f4e7fc700 1 -- 172.21.15.38:0/2418982990 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48012430 con 0x7f5f500cc560 2024-08-15T18:01:50.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.246+0000 7f5f55e12700 1 -- 172.21.15.38:0/2418982990 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f500cc560 msgr2=0x7f5f50160f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:50.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.246+0000 7f5f55e12700 1 --2- 172.21.15.38:0/2418982990 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f500cc560 0x7f5f50160f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009e90 tx=0x7f5f48009780 comp rx=0 tx=0).stop 2024-08-15T18:01:50.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.246+0000 7f5f55e12700 1 -- 172.21.15.38:0/2418982990 shutdown_connections 2024-08-15T18:01:50.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.246+0000 7f5f55e12700 1 --2- 172.21.15.38:0/2418982990 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f500cc560 0x7f5f50160f00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:50.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.246+0000 7f5f55e12700 1 -- 172.21.15.38:0/2418982990 >> 172.21.15.38:0/2418982990 conn(0x7f5f5006e270 msgr2=0x7f5f500b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:50.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.246+0000 7f5f55e12700 1 -- 172.21.15.38:0/2418982990 shutdown_connections 2024-08-15T18:01:50.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.246+0000 7f5f55e12700 1 -- 172.21.15.38:0/2418982990 wait complete. 2024-08-15T18:01:50.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.246+0000 7f5f55e12700 1 Processor -- start 2024-08-15T18:01:50.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.246+0000 7f5f55e12700 1 -- start start 2024-08-15T18:01:50.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f55e12700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f500cc560 0x7f5f50159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:50.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f55e12700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f50159d90 0x7f5f501438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:50.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f55e12700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f5015a160 0x7f5f500c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:50.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f55e12700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f50158d60 con 0x7f5f500cc560 2024-08-15T18:01:50.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f55e12700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f50156e20 con 0x7f5f50159d90 2024-08-15T18:01:50.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f55e12700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f50157100 con 0x7f5f5015a160 2024-08-15T18:01:50.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f4f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f500cc560 0x7f5f50159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:50.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f4f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f500cc560 0x7f5f50159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48332/0 (socket says 172.21.15.38:48332) 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f4f7fe700 1 -- 172.21.15.38:0/2216298033 learned_addr learned my addr 172.21.15.38:0/2216298033 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f4ffff700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f5015a160 0x7f5f500c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f4effd700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f50159d90 0x7f5f501438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f4f7fe700 1 -- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f5015a160 msgr2=0x7f5f500c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f4f7fe700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f5015a160 0x7f5f500c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f4f7fe700 1 -- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f50159d90 msgr2=0x7f5f501438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f4f7fe700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f50159d90 0x7f5f501438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f4f7fe700 1 -- 172.21.15.38:0/2216298033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48009a20 con 0x7f5f500cc560 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f4f7fe700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f500cc560 0x7f5f50159190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000e9a0 tx=0x7f5f40007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f377fe700 1 -- 172.21.15.38:0/2216298033 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40010780 con 0x7f5f500cc560 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f377fe700 1 -- 172.21.15.38:0/2216298033 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f400108e0 con 0x7f5f500cc560 2024-08-15T18:01:50.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f377fe700 1 -- 172.21.15.38:0/2216298033 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40010c80 con 0x7f5f500cc560 2024-08-15T18:01:50.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f55e12700 1 -- 172.21.15.38:0/2216298033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f501573e0 con 0x7f5f500cc560 2024-08-15T18:01:50.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f55e12700 1 -- 172.21.15.38:0/2216298033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f501596d0 con 0x7f5f500cc560 2024-08-15T18:01:50.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.250+0000 7f5f55e12700 1 -- 172.21.15.38:0/2216298033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f50159920 con 0x7f5f500cc560 2024-08-15T18:01:50.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.254+0000 7f5f377fe700 1 -- 172.21.15.38:0/2216298033 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5f40011680 con 0x7f5f500cc560 2024-08-15T18:01:50.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.254+0000 7f5f377fe700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f3c060860 0x7f5f3c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:50.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.254+0000 7f5f377fe700 1 -- 172.21.15.38:0/2216298033 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5448+0+0 (secure 0 0 0) 0x7f5f4007eac0 con 0x7f5f500cc560 2024-08-15T18:01:50.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.254+0000 7f5f4effd700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f3c060860 0x7f5f3c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:50.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.254+0000 7f5f4effd700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f3c060860 0x7f5f3c062d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5f500c3890 tx=0x7f5f44007f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.262+0000 7f5f377fe700 1 -- 172.21.15.38:0/2216298033 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5f4004f010 con 0x7f5f500cc560 2024-08-15T18:01:50.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:50.430+0000 7f5f55e12700 1 -- 172.21.15.38:0/2216298033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000845064.data", "pool2": "cephfs.volume_0000000000845064.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5f500638f0 con 0x7f5f500cc560 2024-08-15T18:01:51.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.134+0000 7f5f377fe700 1 -- 172.21.15.38:0/2216298033 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000845064.data", "pool2": "cephfs.volume_0000000000845064.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000845064.data' removed v43) v1 ==== 241+0+0 (secure 0 0 0) 0x7f5f4001e100 con 0x7f5f500cc560 2024-08-15T18:01:51.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 -- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f3c060860 msgr2=0x7f5f3c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f3c060860 0x7f5f3c062d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5f500c3890 tx=0x7f5f44007f30 comp rx=0 tx=0).stop 2024-08-15T18:01:51.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 -- 172.21.15.38:0/2216298033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f500cc560 msgr2=0x7f5f50159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f500cc560 0x7f5f50159190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000e9a0 tx=0x7f5f40007e10 comp rx=0 tx=0).stop 2024-08-15T18:01:51.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 -- 172.21.15.38:0/2216298033 shutdown_connections 2024-08-15T18:01:51.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5f3c060860 0x7f5f3c062d10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5f5015a160 0x7f5f500c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f50159d90 0x7f5f501438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 --2- 172.21.15.38:0/2216298033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f500cc560 0x7f5f50159190 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 -- 172.21.15.38:0/2216298033 >> 172.21.15.38:0/2216298033 conn(0x7f5f5006e270 msgr2=0x7f5f50152b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:51.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 -- 172.21.15.38:0/2216298033 shutdown_connections 2024-08-15T18:01:51.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.138+0000 7f5f55e12700 1 -- 172.21.15.38:0/2216298033 wait complete. 2024-08-15T18:01:51.144 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs.volume_0000000000845064.data' removed 2024-08-15T18:01:51.157 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-08-15T18:01:51.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.262+0000 7f885f21c700 1 Processor -- start 2024-08-15T18:01:51.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.262+0000 7f885f21c700 1 -- start start 2024-08-15T18:01:51.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.262+0000 7f885f21c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f885813fa50 0x7f88580aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:51.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.262+0000 7f885f21c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88580b0810 con 0x7f885813f680 2024-08-15T18:01:51.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.262+0000 7f885f21c700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88580afc70 con 0x7f88580ad6c0 2024-08-15T18:01:51.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.262+0000 7f885f21c700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88580af060 con 0x7f885813fa50 2024-08-15T18:01:51.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885da19700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f885813fa50 0x7f88580aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:51.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885e21a700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f885813f680 0x7f88580b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40386/0 (socket says 172.21.15.38:40386) 2024-08-15T18:01:51.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885e21a700 1 -- 172.21.15.38:0/1241021530 learned_addr learned my addr 172.21.15.38:0/1241021530 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:51.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885d218700 1 -- 172.21.15.38:0/1241021530 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626032276 0 0) 0x7f88580b0810 con 0x7f885813f680 2024-08-15T18:01:51.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885d218700 1 -- 172.21.15.38:0/1241021530 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8844003570 con 0x7f885813f680 2024-08-15T18:01:51.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885da19700 1 -- 172.21.15.38:0/1241021530 >> v1:172.21.15.104:6789/0 conn(0x7f88580ad6c0 legacy=0x7f885806e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885d218700 1 -- 172.21.15.38:0/1241021530 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328664652 0 0) 0x7f88580afc70 con 0x7f88580ad6c0 2024-08-15T18:01:51.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885da19700 1 -- 172.21.15.38:0/1241021530 >> v1:172.21.15.38:6789/0 conn(0x7f885813f680 legacy=0x7f88580b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885da19700 1 -- 172.21.15.38:0/1241021530 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88580af340 con 0x7f885813fa50 2024-08-15T18:01:51.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885da19700 1 --2- 172.21.15.38:0/1241021530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f885813fa50 0x7f88580aea80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f884c0098e0 tx=0x7f884c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80ef1ad9a0112c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:51.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885d218700 1 -- 172.21.15.38:0/1241021530 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c00b640 con 0x7f885813fa50 2024-08-15T18:01:51.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885d218700 1 -- 172.21.15.38:0/1241021530 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c00b7a0 con 0x7f885813fa50 2024-08-15T18:01:51.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885d218700 1 -- 172.21.15.38:0/1241021530 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c00bb40 con 0x7f885813fa50 2024-08-15T18:01:51.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885f21c700 1 -- 172.21.15.38:0/1241021530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f885813fa50 msgr2=0x7f88580aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885f21c700 1 --2- 172.21.15.38:0/1241021530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f885813fa50 0x7f88580aea80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f884c0098e0 tx=0x7f884c0091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885f21c700 1 -- 172.21.15.38:0/1241021530 shutdown_connections 2024-08-15T18:01:51.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885f21c700 1 --2- 172.21.15.38:0/1241021530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f885813fa50 0x7f88580aea80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885f21c700 1 -- 172.21.15.38:0/1241021530 >> 172.21.15.38:0/1241021530 conn(0x7f88580678a0 msgr2=0x7f8858067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:51.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885f21c700 1 -- 172.21.15.38:0/1241021530 shutdown_connections 2024-08-15T18:01:51.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.266+0000 7f885f21c700 1 -- 172.21.15.38:0/1241021530 wait complete. 2024-08-15T18:01:51.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.270+0000 7f885f21c700 1 Processor -- start 2024-08-15T18:01:51.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.270+0000 7f885f21c700 1 -- start start 2024-08-15T18:01:51.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.270+0000 7f885f21c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88580ad6c0 0x7f8858133590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:51.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.270+0000 7f885f21c700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f885813f680 0x7f885813c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:51.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.270+0000 7f885f21c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f885813fa50 0x7f88581306a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:51.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.270+0000 7f885f21c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885813c670 con 0x7f88580ad6c0 2024-08-15T18:01:51.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.270+0000 7f885f21c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885813c8a0 con 0x7f885813fa50 2024-08-15T18:01:51.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.270+0000 7f885f21c700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8858130be0 con 0x7f885813f680 2024-08-15T18:01:51.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.270+0000 7f885e21a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88580ad6c0 0x7f8858133590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:51.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.270+0000 7f885e21a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88580ad6c0 0x7f8858133590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48354/0 (socket says 172.21.15.38:48354) 2024-08-15T18:01:51.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.270+0000 7f885e21a700 1 -- 172.21.15.38:0/3682858337 learned_addr learned my addr 172.21.15.38:0/3682858337 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:51.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f885da19700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f885813f680 0x7f885813c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:51.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f885ea1b700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f885813fa50 0x7f88581306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:51.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f885e21a700 1 -- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f885813f680 msgr2=0x7f885813c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f885e21a700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f885813f680 0x7f885813c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f885e21a700 1 -- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f885813fa50 msgr2=0x7f88581306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f885e21a700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f885813fa50 0x7f88581306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f885e21a700 1 -- 172.21.15.38:0/3682858337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c009420 con 0x7f88580ad6c0 2024-08-15T18:01:51.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f885e21a700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88580ad6c0 0x7f8858133590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f88540011e0 tx=0x7f8854007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:51.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f884a7fc700 1 -- 172.21.15.38:0/3682858337 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8854005b30 con 0x7f88580ad6c0 2024-08-15T18:01:51.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f884a7fc700 1 -- 172.21.15.38:0/3682858337 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8854005c90 con 0x7f88580ad6c0 2024-08-15T18:01:51.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f885f21c700 1 -- 172.21.15.38:0/3682858337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8858135180 con 0x7f88580ad6c0 2024-08-15T18:01:51.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f884a7fc700 1 -- 172.21.15.38:0/3682858337 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8854007320 con 0x7f88580ad6c0 2024-08-15T18:01:51.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f885f21c700 1 -- 172.21.15.38:0/3682858337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8858135740 con 0x7f88580ad6c0 2024-08-15T18:01:51.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f885f21c700 1 -- 172.21.15.38:0/3682858337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8858064330 con 0x7f88580ad6c0 2024-08-15T18:01:51.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f884a7fc700 1 -- 172.21.15.38:0/3682858337 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8854007690 con 0x7f88580ad6c0 2024-08-15T18:01:51.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f884a7fc700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f883c060860 0x7f883c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:51.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.274+0000 7f884a7fc700 1 -- 172.21.15.38:0/3682858337 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f885407bdb0 con 0x7f88580ad6c0 2024-08-15T18:01:51.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.278+0000 7f885da19700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f883c060860 0x7f883c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:51.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.278+0000 7f885da19700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f883c060860 0x7f883c062d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f884c0093f0 tx=0x7f884c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:51.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.282+0000 7f884a7fc700 1 -- 172.21.15.38:0/3682858337 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f885404c4a0 con 0x7f88580ad6c0 2024-08-15T18:01:51.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.446+0000 7f885f21c700 1 -- 172.21.15.38:0/3682858337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f885806ed70 con 0x7f88580ad6c0 2024-08-15T18:01:51.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.450+0000 7f884a7fc700 1 -- 172.21.15.38:0/3682858337 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f885404fac0 con 0x7f88580ad6c0 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 -- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f883c060860 msgr2=0x7f883c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f883c060860 0x7f883c062d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f884c0093f0 tx=0x7f884c017000 comp rx=0 tx=0).stop 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 -- 172.21.15.38:0/3682858337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88580ad6c0 msgr2=0x7f8858133590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88580ad6c0 0x7f8858133590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f88540011e0 tx=0x7f8854007df0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 -- 172.21.15.38:0/3682858337 shutdown_connections 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f883c060860 0x7f883c062d10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f885813fa50 0x7f88581306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f885813f680 0x7f885813c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 --2- 172.21.15.38:0/3682858337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88580ad6c0 0x7f8858133590 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 -- 172.21.15.38:0/3682858337 >> 172.21.15.38:0/3682858337 conn(0x7f88580678a0 msgr2=0x7f885806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 -- 172.21.15.38:0/3682858337 shutdown_connections 2024-08-15T18:01:51.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.454+0000 7f885f21c700 1 -- 172.21.15.38:0/3682858337 wait complete. 2024-08-15T18:01:51.465 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-08-15T18:01:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.562+0000 7f2752227700 1 Processor -- start 2024-08-15T18:01:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.562+0000 7f2752227700 1 -- start start 2024-08-15T18:01:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.562+0000 7f2752227700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f274c1536f0 0x7f274c15a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.562+0000 7f2752227700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274c153b70 con 0x7f274c153320 2024-08-15T18:01:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.562+0000 7f2752227700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274c15dbd0 con 0x7f274c0cd1b0 2024-08-15T18:01:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.562+0000 7f2752227700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274c0ce430 con 0x7f274c1536f0 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.562+0000 7f2751225700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f274c153320 0x7f274c0ccaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40400/0 (socket says 172.21.15.38:40400) 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.562+0000 7f2751225700 1 -- 172.21.15.38:0/1011734349 learned_addr learned my addr 172.21.15.38:0/1011734349 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.562+0000 7f2751a26700 1 --2- 172.21.15.38:0/1011734349 >> v2:172.21.15.104:3300/0 conn(0x7f274c1536f0 0x7f274c15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f273ffff700 1 -- 172.21.15.38:0/1011734349 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163297291 0 0) 0x7f2748001110 con 0x7f274c153320 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f273ffff700 1 -- 172.21.15.38:0/1011734349 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2738003570 con 0x7f274c153320 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f2751a26700 1 -- 172.21.15.38:0/1011734349 >> v1:172.21.15.104:6789/0 conn(0x7f274c0cd1b0 legacy=0x7f274c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f2751a26700 1 -- 172.21.15.38:0/1011734349 >> v1:172.21.15.38:6789/0 conn(0x7f274c153320 legacy=0x7f274c0ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f2751a26700 1 -- 172.21.15.38:0/1011734349 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c15d370 con 0x7f274c1536f0 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f2751a26700 1 --2- 172.21.15.38:0/1011734349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f274c1536f0 0x7f274c15a190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2744009e90 tx=0x7f2744009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f81c7a084e00b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f273ffff700 1 -- 172.21.15.38:0/1011734349 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274400e070 con 0x7f274c1536f0 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f273ffff700 1 -- 172.21.15.38:0/1011734349 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274400bc40 con 0x7f274c1536f0 2024-08-15T18:01:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f273ffff700 1 -- 172.21.15.38:0/1011734349 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744012430 con 0x7f274c1536f0 2024-08-15T18:01:51.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f2752227700 1 -- 172.21.15.38:0/1011734349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f274c1536f0 msgr2=0x7f274c15a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f2752227700 1 --2- 172.21.15.38:0/1011734349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f274c1536f0 0x7f274c15a190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2744009e90 tx=0x7f2744009780 comp rx=0 tx=0).stop 2024-08-15T18:01:51.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f2752227700 1 -- 172.21.15.38:0/1011734349 shutdown_connections 2024-08-15T18:01:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f2752227700 1 --2- 172.21.15.38:0/1011734349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f274c1536f0 0x7f274c15a190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f2752227700 1 -- 172.21.15.38:0/1011734349 >> 172.21.15.38:0/1011734349 conn(0x7f274c0d00a0 msgr2=0x7f274c0d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f2752227700 1 -- 172.21.15.38:0/1011734349 shutdown_connections 2024-08-15T18:01:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.566+0000 7f2752227700 1 -- 172.21.15.38:0/1011734349 wait complete. 2024-08-15T18:01:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2752227700 1 Processor -- start 2024-08-15T18:01:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2752227700 1 -- start start 2024-08-15T18:01:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2752227700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f274c0cd1b0 0x7f274c0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2752227700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f274c153320 0x7f274c14abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2752227700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f274c1536f0 0x7f274c0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2752227700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0b44a0 con 0x7f274c1536f0 2024-08-15T18:01:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2752227700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0b3890 con 0x7f274c153320 2024-08-15T18:01:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2752227700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0b3b70 con 0x7f274c0cd1b0 2024-08-15T18:01:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2751a26700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f274c1536f0 0x7f274c0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2751a26700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f274c1536f0 0x7f274c0b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48366/0 (socket says 172.21.15.38:48366) 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2751a26700 1 -- 172.21.15.38:0/2519970452 learned_addr learned my addr 172.21.15.38:0/2519970452 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2750a24700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f274c153320 0x7f274c14abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2751225700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f274c0cd1b0 0x7f274c0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2751a26700 1 -- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f274c0cd1b0 msgr2=0x7f274c0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2751a26700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f274c0cd1b0 0x7f274c0b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2751a26700 1 -- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f274c153320 msgr2=0x7f274c14abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2751a26700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f274c153320 0x7f274c14abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2751a26700 1 -- 172.21.15.38:0/2519970452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744009a20 con 0x7f274c1536f0 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2751a26700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f274c1536f0 0x7f274c0b2e40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f27440099f0 tx=0x7f2744019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f273d7fa700 1 -- 172.21.15.38:0/2519970452 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744012e20 con 0x7f274c1536f0 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f273d7fa700 1 -- 172.21.15.38:0/2519970452 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274401c610 con 0x7f274c1536f0 2024-08-15T18:01:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2752227700 1 -- 172.21.15.38:0/2519970452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274c0b3e50 con 0x7f274c1536f0 2024-08-15T18:01:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f273d7fa700 1 -- 172.21.15.38:0/2519970452 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274401c9d0 con 0x7f274c1536f0 2024-08-15T18:01:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.570+0000 7f2752227700 1 -- 172.21.15.38:0/2519970452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274c14b120 con 0x7f274c1536f0 2024-08-15T18:01:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.574+0000 7f2752227700 1 -- 172.21.15.38:0/2519970452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274c0b4650 con 0x7f274c1536f0 2024-08-15T18:01:51.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.574+0000 7f273d7fa700 1 -- 172.21.15.38:0/2519970452 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f274401cd40 con 0x7f274c1536f0 2024-08-15T18:01:51.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.574+0000 7f273d7fa700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2730060890 0x7f2730062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:51.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.574+0000 7f273d7fa700 1 -- 172.21.15.38:0/2519970452 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f274407eb40 con 0x7f274c1536f0 2024-08-15T18:01:51.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.574+0000 7f2751225700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2730060890 0x7f2730062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:51.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.578+0000 7f2751225700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2730060890 0x7f2730062d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2748001270 tx=0x7f2748009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:51.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.578+0000 7f273d7fa700 1 -- 172.21.15.38:0/2519970452 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f274c0b4650 con 0x7f274c1536f0 2024-08-15T18:01:51.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:51.802+0000 7f2752227700 1 -- 172.21.15.38:0/2519970452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7f274c0638f0 con 0x7f274c1536f0 2024-08-15T18:01:52.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.126+0000 7f273d7fa700 1 -- 172.21.15.38:0/2519970452 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v157) v1 ==== 135+0+0 (secure 0 0 0) 0x7f274402b020 con 0x7f274c1536f0 2024-08-15T18:01:52.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 -- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2730060890 msgr2=0x7f2730062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:52.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2730060890 0x7f2730062d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2748001270 tx=0x7f2748009040 comp rx=0 tx=0).stop 2024-08-15T18:01:52.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 -- 172.21.15.38:0/2519970452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f274c1536f0 msgr2=0x7f274c0b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:52.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f274c1536f0 0x7f274c0b2e40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f27440099f0 tx=0x7f2744019000 comp rx=0 tx=0).stop 2024-08-15T18:01:52.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 -- 172.21.15.38:0/2519970452 shutdown_connections 2024-08-15T18:01:52.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2730060890 0x7f2730062d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:52.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f274c1536f0 0x7f274c0b2e40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:52.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f274c153320 0x7f274c14abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:52.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 --2- 172.21.15.38:0/2519970452 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f274c0cd1b0 0x7f274c0b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:52.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 -- 172.21.15.38:0/2519970452 >> 172.21.15.38:0/2519970452 conn(0x7f274c0d00a0 msgr2=0x7f274c15c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:52.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 -- 172.21.15.38:0/2519970452 shutdown_connections 2024-08-15T18:01:52.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:52.130+0000 7f2752227700 1 -- 172.21.15.38:0/2519970452 wait complete. 2024-08-15T18:01:52.156 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ERROR 2024-08-15T18:01:52.156 INFO:tasks.cephfs_test_runner: 2024-08-15T18:01:52.156 INFO:tasks.cephfs_test_runner:====================================================================== 2024-08-15T18:01:52.156 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T18:01:52.156 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2024-08-15T18:01:52.156 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-15T18:01:52.156 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 459, in test_volume_create 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", volname, "--yes-i-really-mean-it") 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_manager.py", line 1631, in run_cluster_cmd 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: r.wait() 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-08-15T18:01:52.157 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-08-15T18:01:52.158 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-08-15T18:01:52.158 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi038 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000845064 --yes-i-really-mean-it' 2024-08-15T18:01:52.159 INFO:tasks.cephfs_test_runner: 2024-08-15T18:01:52.159 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-15T18:01:52.159 INFO:tasks.cephfs_test_runner:Ran 1 test in 167.868s 2024-08-15T18:01:52.159 INFO:tasks.cephfs_test_runner: 2024-08-15T18:01:52.159 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner: 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner:====================================================================== 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 459, in test_volume_create 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", volname, "--yes-i-really-mean-it") 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-08-15T18:01:52.160 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_manager.py", line 1631, in run_cluster_cmd 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner: r.wait() 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi038 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000845064 --yes-i-really-mean-it' 2024-08-15T18:01:52.161 INFO:tasks.cephfs_test_runner: 2024-08-15T18:01:52.161 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T18:01:52.397 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=434350764f0e471da729700c3aa8f152 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T18:01:52.399 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-08-15T18:01:52.408 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-08-15T18:01:52.415 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-08-15T18:01:52.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:52.415 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-08-15T18:01:52.430 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-08-15T18:01:52.431 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-15T18:01:52.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:52.431 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-15T18:01:52.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:52.491 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T18:01:52.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:52.506 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link delete ceph-brx 2024-08-15T18:01:52.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:52.696 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-08-15T18:01:52.704 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-08-15T18:01:52.704 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-15T18:01:52.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:52.705 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-15T18:01:52.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:52.723 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T18:01:52.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T18:01:52.738 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-08-15T18:01:52.905 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-08-15T18:01:52.914 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-08-15T18:01:52.914 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T18:01:53.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.018+0000 7f1027eb3700 1 Processor -- start 2024-08-15T18:01:53.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.018+0000 7f1027eb3700 1 -- start start 2024-08-15T18:01:53.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10200cc2e0 0x7f10200b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f10200cea40 0x7f10200ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10200cf3e0 con 0x7f1020159810 2024-08-15T18:01:53.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10200c5c60 con 0x7f10200cea40 2024-08-15T18:01:53.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10200ce120 con 0x7f10200cc2e0 2024-08-15T18:01:53.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1025c4f700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1020159810 0x7f10200d0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40414/0 (socket says 172.21.15.38:40414) 2024-08-15T18:01:53.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1025c4f700 1 -- 172.21.15.38:0/1052863825 learned_addr learned my addr 172.21.15.38:0/1052863825 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:53.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f102544e700 1 --2- 172.21.15.38:0/1052863825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10200cc2e0 0x7f10200b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1026450700 1 --2- 172.21.15.38:0/1052863825 >> v2:172.21.15.104:3300/0 conn(0x7f10200cea40 0x7f10200ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1024c4d700 1 -- 172.21.15.38:0/1052863825 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294870430 0 0) 0x7f1010001110 con 0x7f1020159810 2024-08-15T18:01:53.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1024c4d700 1 -- 172.21.15.38:0/1052863825 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1008003570 con 0x7f1020159810 2024-08-15T18:01:53.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f102544e700 1 -- 172.21.15.38:0/1052863825 >> v2:172.21.15.104:3300/0 conn(0x7f10200cea40 msgr2=0x7f10200ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:53.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f102544e700 1 --2- 172.21.15.38:0/1052863825 >> v2:172.21.15.104:3300/0 conn(0x7f10200cea40 0x7f10200ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f102544e700 1 -- 172.21.15.38:0/1052863825 >> v1:172.21.15.38:6789/0 conn(0x7f1020159810 legacy=0x7f10200d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f102544e700 1 -- 172.21.15.38:0/1052863825 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10200ce390 con 0x7f10200cc2e0 2024-08-15T18:01:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f102544e700 1 --2- 172.21.15.38:0/1052863825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10200cc2e0 0x7f10200b3270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1018009e60 tx=0x7f1018009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46cb7c1e175689e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1024c4d700 1 -- 172.21.15.38:0/1052863825 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101800e070 con 0x7f10200cc2e0 2024-08-15T18:01:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1024c4d700 1 -- 172.21.15.38:0/1052863825 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101800bab0 con 0x7f10200cc2e0 2024-08-15T18:01:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1024c4d700 1 -- 172.21.15.38:0/1052863825 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101800be70 con 0x7f10200cc2e0 2024-08-15T18:01:53.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 -- 172.21.15.38:0/1052863825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10200cc2e0 msgr2=0x7f10200b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 --2- 172.21.15.38:0/1052863825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10200cc2e0 0x7f10200b3270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1018009e60 tx=0x7f1018009780 comp rx=0 tx=0).stop 2024-08-15T18:01:53.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 -- 172.21.15.38:0/1052863825 shutdown_connections 2024-08-15T18:01:53.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 --2- 172.21.15.38:0/1052863825 >> v2:172.21.15.104:3300/0 conn(0x7f10200cea40 0x7f10200ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 --2- 172.21.15.38:0/1052863825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10200cc2e0 0x7f10200b3270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 -- 172.21.15.38:0/1052863825 >> 172.21.15.38:0/1052863825 conn(0x7f1020067590 msgr2=0x7f10200d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:53.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 -- 172.21.15.38:0/1052863825 shutdown_connections 2024-08-15T18:01:53.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.022+0000 7f1027eb3700 1 -- 172.21.15.38:0/1052863825 wait complete. 2024-08-15T18:01:53.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1027eb3700 1 Processor -- start 2024-08-15T18:01:53.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1027eb3700 1 -- start start 2024-08-15T18:01:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1027eb3700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10200cc2e0 0x7f1020140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1027eb3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10200cea40 0x7f1020156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1027eb3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1020159810 0x7f1020157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1027eb3700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10200c75f0 con 0x7f10200cea40 2024-08-15T18:01:53.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1027eb3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10200c7e90 con 0x7f1020159810 2024-08-15T18:01:53.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1027eb3700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1020157580 con 0x7f10200cc2e0 2024-08-15T18:01:53.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f102544e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10200cea40 0x7f1020156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f102544e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10200cea40 0x7f1020156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48380/0 (socket says 172.21.15.38:48380) 2024-08-15T18:01:53.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f102544e700 1 -- 172.21.15.38:0/3113430632 learned_addr learned my addr 172.21.15.38:0/3113430632 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:53.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1025c4f700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10200cc2e0 0x7f1020140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1026450700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1020159810 0x7f1020157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f102544e700 1 -- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10200cc2e0 msgr2=0x7f1020140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f102544e700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10200cc2e0 0x7f1020140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f102544e700 1 -- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1020159810 msgr2=0x7f1020157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f102544e700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1020159810 0x7f1020157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f102544e700 1 -- 172.21.15.38:0/3113430632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1018009a20 con 0x7f10200cea40 2024-08-15T18:01:53.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1026450700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1020159810 0x7f1020157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:53.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f102544e700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10200cea40 0x7f1020156b00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f10200c5da0 tx=0x7f1018019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1015ffb700 1 -- 172.21.15.38:0/3113430632 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018012b40 con 0x7f10200cea40 2024-08-15T18:01:53.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1015ffb700 1 -- 172.21.15.38:0/3113430632 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1018012ca0 con 0x7f10200cea40 2024-08-15T18:01:53.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1027eb3700 1 -- 172.21.15.38:0/3113430632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10200b4b50 con 0x7f10200cea40 2024-08-15T18:01:53.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1015ffb700 1 -- 172.21.15.38:0/3113430632 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180276c0 con 0x7f10200cea40 2024-08-15T18:01:53.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.026+0000 7f1027eb3700 1 -- 172.21.15.38:0/3113430632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10200c7b20 con 0x7f10200cea40 2024-08-15T18:01:53.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.030+0000 7f1027eb3700 1 -- 172.21.15.38:0/3113430632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10200645c0 con 0x7f10200cea40 2024-08-15T18:01:53.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.030+0000 7f1015ffb700 1 -- 172.21.15.38:0/3113430632 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1018024070 con 0x7f10200cea40 2024-08-15T18:01:53.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.030+0000 7f1015ffb700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f100c060860 0x7f100c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.030+0000 7f1015ffb700 1 -- 172.21.15.38:0/3113430632 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f101807ed60 con 0x7f10200cea40 2024-08-15T18:01:53.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.030+0000 7f1025c4f700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f100c060860 0x7f100c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.034+0000 7f1025c4f700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f100c060860 0x7f100c062d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1010007fd0 tx=0x7f1010007bf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.034+0000 7f1015ffb700 1 -- 172.21.15.38:0/3113430632 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f101804f4a0 con 0x7f10200cea40 2024-08-15T18:01:53.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.202+0000 7f1027eb3700 1 -- 172.21.15.38:0/3113430632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1020071590 con 0x7f10200cea40 2024-08-15T18:01:53.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.202+0000 7f1015ffb700 1 -- 172.21.15.38:0/3113430632 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12395 (secure 0 0 0) 0x7f1018052ac0 con 0x7f10200cea40 2024-08-15T18:01:53.206 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:53.206 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":43,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T18:01:51.113831+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:6834/875828080":"2024-08-16T18:01:48.089417+0000","172.21.15.104:6837/1865762791":"2024-08-16T18:01:41.797233+0000","172.21.15.104:6835/875828080":"2024-08-16T18:01:48.089417+0000","172.21.15.104:6836/1865762791":"2024-08-16T18:01:41.797233+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T18:01:53.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 -- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f100c060860 msgr2=0x7f100c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f100c060860 0x7f100c062d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1010007fd0 tx=0x7f1010007bf0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 -- 172.21.15.38:0/3113430632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10200cea40 msgr2=0x7f1020156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10200cea40 0x7f1020156b00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f10200c5da0 tx=0x7f1018019000 comp rx=0 tx=0).stop 2024-08-15T18:01:53.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 -- 172.21.15.38:0/3113430632 shutdown_connections 2024-08-15T18:01:53.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f100c060860 0x7f100c062d10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1020159810 0x7f1020157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10200cea40 0x7f1020156b00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 --2- 172.21.15.38:0/3113430632 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10200cc2e0 0x7f1020140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 -- 172.21.15.38:0/3113430632 >> 172.21.15.38:0/3113430632 conn(0x7f1020067590 msgr2=0x7f10200693a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:53.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 -- 172.21.15.38:0/3113430632 shutdown_connections 2024-08-15T18:01:53.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.206+0000 7f1027eb3700 1 -- 172.21.15.38:0/3113430632 wait complete. 2024-08-15T18:01:53.221 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-15T18:01:53.221 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T18:01:53.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.322+0000 7f761d491700 1 Processor -- start 2024-08-15T18:01:53.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f761d491700 1 -- start start 2024-08-15T18:01:53.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f761d491700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76180c9dd0 0x7f76180680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f761d491700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f761815a4c0 0x7f76180685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f761d491700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f761815d830 con 0x7f761815a0f0 2024-08-15T18:01:53.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f761d491700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f761815cf00 con 0x7f761815a4c0 2024-08-15T18:01:53.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f761d491700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761815d1e0 con 0x7f76180c9dd0 2024-08-15T18:01:53.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f7617fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f761815a0f0 0x7f76180c96c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40416/0 (socket says 172.21.15.38:40416) 2024-08-15T18:01:53.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f7617fff700 1 -- 172.21.15.38:0/121305225 learned_addr learned my addr 172.21.15.38:0/121305225 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:53.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f761cc90700 1 --2- 172.21.15.38:0/121305225 >> v2:172.21.15.104:3300/0 conn(0x7f761815a4c0 0x7f76180685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f760ffff700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76180c9dd0 0x7f76180680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257380001 0 0) 0x7f761815d830 con 0x7f761815a0f0 2024-08-15T18:01:53.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7604003570 con 0x7f761815a0f0 2024-08-15T18:01:53.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 902556804 0 0) 0x7f7604003570 con 0x7f761815a0f0 2024-08-15T18:01:53.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76180c9dd0 msgr2=0x7f76180680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:53.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76180c9dd0 0x7f76180680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 >> v2:172.21.15.104:3300/0 conn(0x7f761815a4c0 msgr2=0x7f76180685f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:53.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 --2- 172.21.15.38:0/121305225 >> v2:172.21.15.104:3300/0 conn(0x7f761815a4c0 0x7f76180685f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761815d4c0 con 0x7f761815a0f0 2024-08-15T18:01:53.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f7608001470 con 0x7f761815a0f0 2024-08-15T18:01:53.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 >> v1:172.21.15.38:6789/0 conn(0x7f761815a0f0 legacy=0x7f76180c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76180c9dd0 0x7f7604003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7604003fe0 0x7f7604004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7604004d50 0x7f76040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761815cf00 con 0x7f7604003fe0 2024-08-15T18:01:53.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761815d1e0 con 0x7f7604004d50 2024-08-15T18:01:53.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.326+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760400ba90 con 0x7f76180c9dd0 2024-08-15T18:01:53.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f760ffff700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7604003fe0 0x7f7604004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f761cc90700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7604004d50 0x7f76040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f7617fff700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76180c9dd0 0x7f7604003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f760ffff700 1 -- 172.21.15.38:0/121305225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76180c9dd0 msgr2=0x7f7604003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:53.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f760ffff700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76180c9dd0 0x7f7604003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f760ffff700 1 -- 172.21.15.38:0/121305225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7604004d50 msgr2=0x7f76040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:53.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f760ffff700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7604004d50 0x7f76040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f760ffff700 1 -- 172.21.15.38:0/121305225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7604003570 con 0x7f7604003fe0 2024-08-15T18:01:53.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f760ffff700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7604003fe0 0x7f7604004810 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7600009c00 tx=0x7f76000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b7ae216534e91e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760000e6b0 con 0x7f7604003fe0 2024-08-15T18:01:53.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760000e810 con 0x7f7604003fe0 2024-08-15T18:01:53.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f76177fe700 1 -- 172.21.15.38:0/121305225 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760000ebd0 con 0x7f7604003fe0 2024-08-15T18:01:53.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f761d491700 1 -- 172.21.15.38:0/121305225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7604003fe0 msgr2=0x7f7604004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f761d491700 1 --2- 172.21.15.38:0/121305225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7604003fe0 0x7f7604004810 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7600009c00 tx=0x7f76000091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f761cc90700 1 -- 172.21.15.38:0/121305225 reap_dead start 2024-08-15T18:01:53.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f761d491700 1 -- 172.21.15.38:0/121305225 shutdown_connections 2024-08-15T18:01:53.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f761d491700 1 -- 172.21.15.38:0/121305225 >> 172.21.15.38:0/121305225 conn(0x7f76180ccfc0 msgr2=0x7f76180c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:53.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f761d491700 1 -- 172.21.15.38:0/121305225 shutdown_connections 2024-08-15T18:01:53.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f761d491700 1 -- 172.21.15.38:0/121305225 wait complete. 2024-08-15T18:01:53.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f761d491700 1 Processor -- start 2024-08-15T18:01:53.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.330+0000 7f761d491700 1 -- start start 2024-08-15T18:01:53.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761d491700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7618151c80 0x7f7618152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761d491700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7618154070 0x7f761816d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761d491700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f761814af00 0x7f761814b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761d491700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7618156c60 con 0x7f761814af00 2024-08-15T18:01:53.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761d491700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7618154550 con 0x7f7618151c80 2024-08-15T18:01:53.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761d491700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7618154830 con 0x7f7618154070 2024-08-15T18:01:53.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761cc90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f761814af00 0x7f761814b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761cc90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f761814af00 0x7f761814b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48396/0 (socket says 172.21.15.38:48396) 2024-08-15T18:01:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761cc90700 1 -- 172.21.15.38:0/2284950051 learned_addr learned my addr 172.21.15.38:0/2284950051 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f7617fff700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7618151c80 0x7f7618152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f760ffff700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7618154070 0x7f761816d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761cc90700 1 -- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7618154070 msgr2=0x7f761816d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761cc90700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7618154070 0x7f761816d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761cc90700 1 -- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7618151c80 msgr2=0x7f7618152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761cc90700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7618151c80 0x7f7618152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761cc90700 1 -- 172.21.15.38:0/2284950051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7618154b10 con 0x7f761814af00 2024-08-15T18:01:53.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761cc90700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f761814af00 0x7f761814b3a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f76180b0390 tx=0x7f761000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f7614ff9700 1 -- 172.21.15.38:0/2284950051 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761000e040 con 0x7f761814af00 2024-08-15T18:01:53.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f7614ff9700 1 -- 172.21.15.38:0/2284950051 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761000f040 con 0x7f761814af00 2024-08-15T18:01:53.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761d491700 1 -- 172.21.15.38:0/2284950051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7618154df0 con 0x7f761814af00 2024-08-15T18:01:53.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f7614ff9700 1 -- 172.21.15.38:0/2284950051 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761000fd70 con 0x7f761814af00 2024-08-15T18:01:53.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761d491700 1 -- 172.21.15.38:0/2284950051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761816dc60 con 0x7f761814af00 2024-08-15T18:01:53.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.334+0000 7f761d491700 1 -- 172.21.15.38:0/2284950051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76180638f0 con 0x7f761814af00 2024-08-15T18:01:53.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.338+0000 7f7614ff9700 1 -- 172.21.15.38:0/2284950051 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7610015640 con 0x7f761814af00 2024-08-15T18:01:53.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.338+0000 7f7614ff9700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f75f8060860 0x7f75f8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.338+0000 7f7614ff9700 1 -- 172.21.15.38:0/2284950051 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f761007d0e0 con 0x7f761814af00 2024-08-15T18:01:53.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.338+0000 7f7617fff700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f75f8060860 0x7f75f8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.338+0000 7f7617fff700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f75f8060860 0x7f75f8062d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7608001d20 tx=0x7f7608001e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.346+0000 7f7614ff9700 1 -- 172.21.15.38:0/2284950051 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f761004d7d0 con 0x7f761814af00 2024-08-15T18:01:53.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.502+0000 7f761d491700 1 -- 172.21.15.38:0/2284950051 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f761806ea50 con 0x7f75f8060860 2024-08-15T18:01:53.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.506+0000 7f7614ff9700 1 -- 172.21.15.38:0/2284950051 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48877 (secure 0 0 0) 0x7f761806ea50 con 0x7f75f8060860 2024-08-15T18:01:53.511 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:53.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 -- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f75f8060860 msgr2=0x7f75f8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f75f8060860 0x7f75f8062d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7608001d20 tx=0x7f7608001e90 comp rx=0 tx=0).stop 2024-08-15T18:01:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 -- 172.21.15.38:0/2284950051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f761814af00 msgr2=0x7f761814b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f761814af00 0x7f761814b3a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f76180b0390 tx=0x7f761000b040 comp rx=0 tx=0).stop 2024-08-15T18:01:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 -- 172.21.15.38:0/2284950051 shutdown_connections 2024-08-15T18:01:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f75f8060860 0x7f75f8062d10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f761814af00 0x7f761814b3a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7618154070 0x7f761816d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 --2- 172.21.15.38:0/2284950051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7618151c80 0x7f7618152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 -- 172.21.15.38:0/2284950051 >> 172.21.15.38:0/2284950051 conn(0x7f76180ccfc0 msgr2=0x7f76180c8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 -- 172.21.15.38:0/2284950051 shutdown_connections 2024-08-15T18:01:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.510+0000 7f761d491700 1 -- 172.21.15.38:0/2284950051 wait complete. 2024-08-15T18:01:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-08-15T18:01:53.529 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-08-15T18:01:52.246011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":2408792,"kb_used_data":4960,"kb_used_omap":13,"kb_used_meta":2403698,"kb_avail":752565928,"statfs":{"total":773094113280,"available":770627510272,"internally_reserved":0,"allocated":5079040,"data_stored":2797077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2461387456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001682"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-08-15T18:01:44.112384+0000","last_change":"2024-08-15T17:58:24.394014+0000","last_active":"2024-08-15T18:01:44.112384+0000","last_peered":"2024-08-15T18:01:44.112384+0000","last_clean":"2024-08-15T18:01:44.112384+0000","last_became_active":"2024-08-15T17:58:24.391735+0000","last_became_peered":"2024-08-15T17:58:24.391735+0000","last_unstale":"2024-08-15T18:01:44.112384+0000","last_undegraded":"2024-08-15T18:01:44.112384+0000","last_fullsized":"2024-08-15T18:01:44.112384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_clean_scrub_stamp":"2024-08-15T17:58:22.996335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:31.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302552,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069288,"statfs":{"total":96636764160,"available":96326950912,"internally_reserved":0,"allocated":1056768,"data_stored":782443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92600000000000005,"15min":0.92800000000000005},"min":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1919999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.86299999999999999,"15min":0.871},"min":{"1min":0.64300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.208,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0589999999999999}]},{"osd":1,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91800000000000004,"15min":0.90700000000000003},"min":{"1min":0.67300000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89200000000000002,"15min":0.875},"min":{"1min":0.76900000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0880000000000001,"5min":1.165,"15min":1.165},"last":0.746}]},{"osd":2,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.73099999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.262,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.155},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1,"15min":1.022},"min":{"1min":0.65400000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.111,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.94599999999999995,"15min":0.93999999999999995},"min":{"1min":0.73799999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.01},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0880000000000001,"15min":1.083},"min":{"1min":0.86199999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3160000000000001,"5min":1.415,"15min":1.415},"last":1.26}]},{"osd":4,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.93999999999999995,"15min":0.96199999999999997},"min":{"1min":0.622,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.21,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.034,"5min":1.0529999999999999,"15min":1.0780000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2350000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.329}]},{"osd":5,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.105,"15min":1.1319999999999999},"min":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3080000000000001,"5min":1.599,"15min":1.599},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.131,"15min":1.1519999999999999},"min":{"1min":0.78400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3660000000000001,"5min":1.486,"15min":1.486},"last":0.98399999999999999}]},{"osd":6,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1970000000000001,"15min":1.2170000000000001},"min":{"1min":0.85299999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.379,"5min":1.631,"15min":1.631},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1850000000000001,"15min":1.1950000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.49,"5min":1.518,"15min":1.518},"last":1.498}]}]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":466944,"data_stored":192645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85199999999999998,"15min":0.81999999999999995},"min":{"1min":0.58999999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.224,"5min":1.226,"15min":1.226},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.91400000000000003,"15min":0.93600000000000005},"min":{"1min":0.622,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1970000000000001,"5min":1.516,"15min":1.516},"last":1.155}]},{"osd":1,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88600000000000001,"15min":0.90800000000000003},"min":{"1min":0.66000000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.054,"5min":1.333,"15min":1.333},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.99299999999999999,"15min":1.006},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2669999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.84799999999999998}]},{"osd":2,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.98299999999999998,"15min":1.0009999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.274,"5min":1.369,"15min":1.369},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.998,"5min":1.0109999999999999,"15min":1.02},"min":{"1min":0.80600000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2609999999999999,"5min":1.409,"15min":1.409},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.67500000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1419999999999999,"5min":1.302,"15min":1.302},"last":1.083},{"interface":"front","average":{"1min":1.032,"5min":1.022,"15min":1.026},"min":{"1min":0.83599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.26,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.95699999999999996,"15min":0.96599999999999997},"min":{"1min":0.63400000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.147,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.105,"15min":1.131},"min":{"1min":0.73299999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.306,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":1.2230000000000001}]},{"osd":5,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.876,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.234,"5min":1.345,"15min":1.345},"last":1.129},{"interface":"front","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.161},"min":{"1min":0.999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.3240000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.111}]},{"osd":7,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1970000000000001,"15min":1.232},"min":{"1min":0.97399999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.377,"5min":1.631,"15min":1.631},"last":1.28},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.159,"15min":1.171},"min":{"1min":1,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4770000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.1950000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":471040,"data_stored":193537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2310000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.95199999999999996,"15min":0.96699999999999997},"min":{"1min":0.65300000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2749999999999999,"5min":1.371,"15min":1.371},"last":0.93100000000000005}]},{"osd":1,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.96699999999999997,"15min":0.98299999999999998},"min":{"1min":0.64300000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1060000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.98699999999999999,"15min":1.0309999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.167,"5min":1.708,"15min":1.708},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.998,"15min":1.03},"min":{"1min":0.72799999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2629999999999999,"5min":1.514,"15min":1.514},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0429999999999999,"15min":1.0669999999999999},"min":{"1min":0.73099999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.2090000000000001,"5min":1.476,"15min":1.476},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.1020000000000001,"15min":1.1399999999999999},"min":{"1min":0.82099999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.335,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":1.091,"15min":1.135},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3149999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.81299999999999994}]},{"osd":5,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.012,"15min":1.054},"min":{"1min":0.68500000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.208,"5min":1.419,"15min":1.419},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.137,"15min":1.1990000000000001},"min":{"1min":0.69699999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.3120000000000001,"5min":1.621,"15min":1.621},"last":1.087}]},{"osd":6,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.121,"15min":1.167},"min":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3240000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.198,"15min":1.264},"min":{"1min":0.82899999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.2629999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.0620000000000001}]},{"osd":7,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2789999999999999,"15min":1.353},"min":{"1min":0.86799999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3799999999999999,"5min":1.772,"15min":1.772},"last":1.139},{"interface":"front","average":{"1min":1.171,"5min":1.2989999999999999,"15min":1.345},"min":{"1min":0.91700000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.391,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0229999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301980,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069860,"statfs":{"total":96636764160,"available":96327536640,"internally_reserved":0,"allocated":475136,"data_stored":198256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.71099999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.248,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.04,"15min":1.042},"min":{"1min":0.75600000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2470000000000001,"5min":1.383,"15min":1.383},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.66700000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96499999999999997,"15min":0.95699999999999996},"min":{"1min":0.71699999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2070000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95399999999999996,"15min":0.96099999999999997},"min":{"1min":0.76700000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.177,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.996,"15min":1.012},"min":{"1min":0.69699999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.2949999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.016}]},{"osd":3,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.92400000000000004,"15min":0.92500000000000004},"min":{"1min":0.73399999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.01,"15min":0.98799999999999999},"min":{"1min":0.80900000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.96999999999999997}]},{"osd":4,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.036,"5min":1.0189999999999999,"15min":1.0249999999999999},"min":{"1min":0.73099999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.99199999999999999,"15min":0.99399999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1950000000000001,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.117,"15min":1.1080000000000001},"min":{"1min":0.997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.075}]},{"osd":7,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.071,"15min":1.075},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.198,"5min":1.1679999999999999,"15min":1.165},"min":{"1min":0.997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.3420000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.101}]}]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.998,"15min":0.998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.99299999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.879}]},{"osd":1,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.53000000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.03,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.69599999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.70699999999999996}]},{"osd":2,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.872,"15min":0.88200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.214,"5min":1.74,"15min":1.74},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.97899999999999998,"15min":0.99299999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.089,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.1659999999999999}]},{"osd":4,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1100000000000001,"15min":1.1319999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.444,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.252},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.022,"15min":1.0309999999999999},"min":{"1min":0.83599999999999997,"5min":0.755,"15min":0.755},"max":{"1min":1.4239999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.214}]},{"osd":5,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.167,"15min":1.181},"min":{"1min":0.76700000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5649999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.411},{"interface":"front","average":{"1min":1.028,"5min":1.1220000000000001,"15min":1.1699999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.448,"5min":1.581,"15min":1.581},"last":1.325}]},{"osd":6,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1819999999999999,"15min":1.204},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5529999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.296},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6160000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.5009999999999999}]},{"osd":7,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2030000000000001,"15min":1.2250000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5589999999999999,"5min":2.073,"15min":2.073},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2250000000000001,"15min":1.244},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.4890000000000001,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.3520000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302016,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069824,"statfs":{"total":96636764160,"available":96327499776,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.105,"5min":1.204,"15min":1.204},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.61099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0589999999999999,"5min":1.111,"15min":1.111},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.626,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1140000000000001,"5min":1.127,"15min":1.127},"last":1.038},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2270000000000001,"5min":1.278,"15min":1.278},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.57899999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0009999999999999,"5min":1.073,"15min":1.073},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.93000000000000005},"min":{"1min":0.73699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2090000000000001,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.117}]},{"osd":4,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.0089999999999999,"15min":0.996},"min":{"1min":0.77500000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.296,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.083,"15min":1.0900000000000001},"min":{"1min":0.82599999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.335,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.1359999999999999,"15min":1.155},"min":{"1min":0.85499999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.292,"5min":1.573,"15min":1.573},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.091,"5min":1.099,"15min":1.107},"min":{"1min":0.87,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3300000000000001,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.137,"15min":1.159},"min":{"1min":0.81399999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.3480000000000001,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.141,"5min":1.1739999999999999,"15min":1.1830000000000001},"min":{"1min":0.84499999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.399,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.238}]},{"osd":7,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2110000000000001,"15min":1.218},"min":{"1min":0.96499999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.202,"5min":1.2290000000000001,"15min":1.244},"min":{"1min":1.0129999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4259999999999999,"5min":1.861,"15min":1.861},"last":1.073}]}]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.52300000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.83699999999999997}]},{"osd":2,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.443,"15min":0.443},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.749},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90600000000000003,"15min":0.89100000000000001},"min":{"1min":0.70699999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":1.155},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90100000000000002,"15min":0.92400000000000004},"min":{"1min":0.66300000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0700000000000001,"5min":1.778,"15min":1.778},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.069,"15min":1.093},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4199999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0529999999999999,"15min":1.056},"min":{"1min":0.79200000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.4750000000000001,"5min":1.825,"15min":1.825},"last":1.52}]},{"osd":5,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.089,"15min":1.1040000000000001},"min":{"1min":0.879,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.2909999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.323},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.143,"15min":1.1559999999999999},"min":{"1min":0.85099999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.409,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.2310000000000001}]},{"osd":6,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1339999999999999,"15min":1.1319999999999999},"min":{"1min":0.90800000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.425,"5min":1.734,"15min":1.734},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1639999999999999,"15min":1.1599999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.478,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.4970000000000001}]},{"osd":7,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.1970000000000001,"15min":1.1930000000000001},"min":{"1min":1.0169999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.5920000000000001,"5min":1.861,"15min":1.861},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.95199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.7210000000000001,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.028}]}]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302592,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069248,"statfs":{"total":96636764160,"available":96326909952,"internally_reserved":0,"allocated":1097728,"data_stored":800325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66100000000000003,"15min":0.67900000000000005},"min":{"1min":0.497,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82199999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.72399999999999998}]},{"osd":2,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.53500000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.2310000000000001,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":0.90700000000000003,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83399999999999996,"15min":0.82999999999999996},"min":{"1min":0.66000000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.90800000000000003,"15min":0.90400000000000003},"min":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.333,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.056,"15min":1.044},"min":{"1min":0.80000000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3899999999999999,"5min":1.919,"15min":1.919},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.006,"15min":1.008},"min":{"1min":0.79800000000000004,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.0349999999999999}]},{"osd":5,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.022,"15min":1.0169999999999999},"min":{"1min":0.82299999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.1739999999999999,"15min":1.1990000000000001},"min":{"1min":0.84299999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4399999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.194,"15min":1.202},"min":{"1min":0.89600000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1619999999999999,"15min":1.1539999999999999},"min":{"1min":0.95199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.502,"5min":1.841,"15min":1.841},"last":1.115}]},{"osd":7,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1459999999999999,"15min":1.1359999999999999},"min":{"1min":0.90600000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2509999999999999,"15min":1.2529999999999999},"min":{"1min":1.0369999999999999,"5min":0.878,"15min":0.878},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.379}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T18:01:53.530 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T18:01:53.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.634+0000 7fbaaac28700 1 Processor -- start 2024-08-15T18:01:53.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.634+0000 7fbaaac28700 1 -- start start 2024-08-15T18:01:53.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbaa4155d20 0x7fbaa40b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbaa40ad7c0 0x7fbaa40aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa40af2b0 con 0x7fbaa406df10 2024-08-15T18:01:53.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa40adc10 con 0x7fbaa4155d20 2024-08-15T18:01:53.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa40af590 con 0x7fbaa40ad7c0 2024-08-15T18:01:53.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa9c26700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbaa4155d20 0x7fbaa40b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa9c26700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbaa4155d20 0x7fbaa40b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48404/0 (socket says 172.21.15.38:48404) 2024-08-15T18:01:53.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa9c26700 1 -- 172.21.15.38:0/2181042302 learned_addr learned my addr 172.21.15.38:0/2181042302 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:53.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa9425700 1 --2- 172.21.15.38:0/2181042302 >> v2:172.21.15.104:3300/0 conn(0x7fbaa40ad7c0 0x7fbaa40aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa9c26700 1 -- 172.21.15.38:0/2181042302 >> v2:172.21.15.104:3300/0 conn(0x7fbaa40ad7c0 msgr2=0x7fbaa40aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:53.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa9c26700 1 --2- 172.21.15.38:0/2181042302 >> v2:172.21.15.104:3300/0 conn(0x7fbaa40ad7c0 0x7fbaa40aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa9c26700 1 -- 172.21.15.38:0/2181042302 >> v1:172.21.15.104:6789/0 conn(0x7fbaa406df10 legacy=0x7fbaa406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa9c26700 1 -- 172.21.15.38:0/2181042302 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa40af870 con 0x7fbaa4155d20 2024-08-15T18:01:53.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa9c26700 1 --2- 172.21.15.38:0/2181042302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa4155d20 0x7fbaa40b0350 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbaa406fb30 tx=0x7fbaa0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85fceb2b279d0ff server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa8c24700 1 -- 172.21.15.38:0/2181042302 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0009e70 con 0x7fbaa4155d20 2024-08-15T18:01:53.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa8c24700 1 -- 172.21.15.38:0/2181042302 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa000b860 con 0x7fbaa4155d20 2024-08-15T18:01:53.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaa8c24700 1 -- 172.21.15.38:0/2181042302 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa000bc00 con 0x7fbaa4155d20 2024-08-15T18:01:53.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 -- 172.21.15.38:0/2181042302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa4155d20 msgr2=0x7fbaa40b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 --2- 172.21.15.38:0/2181042302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa4155d20 0x7fbaa40b0350 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbaa406fb30 tx=0x7fbaa0009250 comp rx=0 tx=0).stop 2024-08-15T18:01:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 -- 172.21.15.38:0/2181042302 shutdown_connections 2024-08-15T18:01:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 --2- 172.21.15.38:0/2181042302 >> v2:172.21.15.104:3300/0 conn(0x7fbaa40ad7c0 0x7fbaa40aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 --2- 172.21.15.38:0/2181042302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa4155d20 0x7fbaa40b0350 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 -- 172.21.15.38:0/2181042302 >> 172.21.15.38:0/2181042302 conn(0x7fbaa40678d0 msgr2=0x7fbaa4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 -- 172.21.15.38:0/2181042302 shutdown_connections 2024-08-15T18:01:53.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.638+0000 7fbaaac28700 1 -- 172.21.15.38:0/2181042302 wait complete. 2024-08-15T18:01:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaac28700 1 Processor -- start 2024-08-15T18:01:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaac28700 1 -- start start 2024-08-15T18:01:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaac28700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbaa406df10 0x7fbaa4143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaac28700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbaa40ad7c0 0x7fbaa4146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaac28700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa4155d20 0x7fbaa4145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaac28700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa41451c0 con 0x7fbaa4155d20 2024-08-15T18:01:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaac28700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa41445b0 con 0x7fbaa406df10 2024-08-15T18:01:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaac28700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa4144890 con 0x7fbaa40ad7c0 2024-08-15T18:01:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaa427700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa4155d20 0x7fbaa4145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaa427700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa4155d20 0x7fbaa4145650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48418/0 (socket says 172.21.15.38:48418) 2024-08-15T18:01:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaa427700 1 -- 172.21.15.38:0/3892306212 learned_addr learned my addr 172.21.15.38:0/3892306212 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaa9c26700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbaa406df10 0x7fbaa4143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaa9425700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbaa40ad7c0 0x7fbaa4146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaa427700 1 -- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbaa40ad7c0 msgr2=0x7fbaa4146f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaa427700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbaa40ad7c0 0x7fbaa4146f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaa427700 1 -- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbaa406df10 msgr2=0x7fbaa4143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaa427700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbaa406df10 0x7fbaa4143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaa427700 1 -- 172.21.15.38:0/3892306212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa00094f0 con 0x7fbaa4155d20 2024-08-15T18:01:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaa9c26700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbaa406df10 0x7fbaa4143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaa427700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa4155d20 0x7fbaa4145650 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00c840 tx=0x7fba9c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fba99ffb700 1 -- 172.21.15.38:0/3892306212 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c012630 con 0x7fbaa4155d20 2024-08-15T18:01:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fba99ffb700 1 -- 172.21.15.38:0/3892306212 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c0127b0 con 0x7fbaa4155d20 2024-08-15T18:01:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaac28700 1 -- 172.21.15.38:0/3892306212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa4144b70 con 0x7fbaa4155d20 2024-08-15T18:01:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fba99ffb700 1 -- 172.21.15.38:0/3892306212 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c012b70 con 0x7fbaa4155d20 2024-08-15T18:01:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.642+0000 7fbaaac28700 1 -- 172.21.15.38:0/3892306212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa4147480 con 0x7fbaa4155d20 2024-08-15T18:01:53.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.646+0000 7fbaaac28700 1 -- 172.21.15.38:0/3892306212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa4145370 con 0x7fbaa4155d20 2024-08-15T18:01:53.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.646+0000 7fba99ffb700 1 -- 172.21.15.38:0/3892306212 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fba9c003030 con 0x7fbaa4155d20 2024-08-15T18:01:53.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.646+0000 7fba99ffb700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba88060890 0x7fba88062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.646+0000 7fba99ffb700 1 -- 172.21.15.38:0/3892306212 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fba9c07c890 con 0x7fbaa4155d20 2024-08-15T18:01:53.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.646+0000 7fbaa9c26700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba88060890 0x7fba88062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.646+0000 7fbaa9c26700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba88060890 0x7fba88062d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbaa406fb30 tx=0x7fbaa0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.658+0000 7fba99ffb700 1 -- 172.21.15.38:0/3892306212 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbaa4145370 con 0x7fbaa4155d20 2024-08-15T18:01:53.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.822+0000 7fbaaac28700 1 -- 172.21.15.38:0/3892306212 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbaa40643a0 con 0x7fba88060890 2024-08-15T18:01:53.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.826+0000 7fba99ffb700 1 -- 172.21.15.38:0/3892306212 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48877 (secure 0 0 0) 0x7fbaa40643a0 con 0x7fba88060890 2024-08-15T18:01:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 -- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba88060890 msgr2=0x7fba88062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba88060890 0x7fba88062d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbaa406fb30 tx=0x7fbaa0017000 comp rx=0 tx=0).stop 2024-08-15T18:01:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 -- 172.21.15.38:0/3892306212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa4155d20 msgr2=0x7fbaa4145650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa4155d20 0x7fbaa4145650 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00c840 tx=0x7fba9c001670 comp rx=0 tx=0).stop 2024-08-15T18:01:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 -- 172.21.15.38:0/3892306212 shutdown_connections 2024-08-15T18:01:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fba88060890 0x7fba88062d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa4155d20 0x7fbaa4145650 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbaa40ad7c0 0x7fbaa4146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 --2- 172.21.15.38:0/3892306212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbaa406df10 0x7fbaa4143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 -- 172.21.15.38:0/3892306212 >> 172.21.15.38:0/3892306212 conn(0x7fbaa40678d0 msgr2=0x7fbaa4063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 -- 172.21.15.38:0/3892306212 shutdown_connections 2024-08-15T18:01:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.830+0000 7fbaaac28700 1 -- 172.21.15.38:0/3892306212 wait complete. 2024-08-15T18:01:53.835 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-08-15T18:01:53.847 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-08-15T18:01:52.246011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":2408792,"kb_used_data":4960,"kb_used_omap":13,"kb_used_meta":2403698,"kb_avail":752565928,"statfs":{"total":773094113280,"available":770627510272,"internally_reserved":0,"allocated":5079040,"data_stored":2797077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2461387456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001682"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-08-15T18:01:44.112384+0000","last_change":"2024-08-15T17:58:24.394014+0000","last_active":"2024-08-15T18:01:44.112384+0000","last_peered":"2024-08-15T18:01:44.112384+0000","last_clean":"2024-08-15T18:01:44.112384+0000","last_became_active":"2024-08-15T17:58:24.391735+0000","last_became_peered":"2024-08-15T17:58:24.391735+0000","last_unstale":"2024-08-15T18:01:44.112384+0000","last_undegraded":"2024-08-15T18:01:44.112384+0000","last_fullsized":"2024-08-15T18:01:44.112384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_clean_scrub_stamp":"2024-08-15T17:58:22.996335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:31.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302552,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069288,"statfs":{"total":96636764160,"available":96326950912,"internally_reserved":0,"allocated":1056768,"data_stored":782443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92600000000000005,"15min":0.92800000000000005},"min":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1919999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.86299999999999999,"15min":0.871},"min":{"1min":0.64300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.208,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0589999999999999}]},{"osd":1,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91800000000000004,"15min":0.90700000000000003},"min":{"1min":0.67300000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89200000000000002,"15min":0.875},"min":{"1min":0.76900000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0880000000000001,"5min":1.165,"15min":1.165},"last":0.746}]},{"osd":2,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.73099999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.262,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.155},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1,"15min":1.022},"min":{"1min":0.65400000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.111,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.94599999999999995,"15min":0.93999999999999995},"min":{"1min":0.73799999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.01},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0880000000000001,"15min":1.083},"min":{"1min":0.86199999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3160000000000001,"5min":1.415,"15min":1.415},"last":1.26}]},{"osd":4,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.93999999999999995,"15min":0.96199999999999997},"min":{"1min":0.622,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.21,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.034,"5min":1.0529999999999999,"15min":1.0780000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2350000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.329}]},{"osd":5,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.105,"15min":1.1319999999999999},"min":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3080000000000001,"5min":1.599,"15min":1.599},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.131,"15min":1.1519999999999999},"min":{"1min":0.78400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3660000000000001,"5min":1.486,"15min":1.486},"last":0.98399999999999999}]},{"osd":6,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1970000000000001,"15min":1.2170000000000001},"min":{"1min":0.85299999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.379,"5min":1.631,"15min":1.631},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1850000000000001,"15min":1.1950000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.49,"5min":1.518,"15min":1.518},"last":1.498}]}]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":466944,"data_stored":192645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85199999999999998,"15min":0.81999999999999995},"min":{"1min":0.58999999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.224,"5min":1.226,"15min":1.226},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.91400000000000003,"15min":0.93600000000000005},"min":{"1min":0.622,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1970000000000001,"5min":1.516,"15min":1.516},"last":1.155}]},{"osd":1,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88600000000000001,"15min":0.90800000000000003},"min":{"1min":0.66000000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.054,"5min":1.333,"15min":1.333},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.99299999999999999,"15min":1.006},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2669999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.84799999999999998}]},{"osd":2,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.98299999999999998,"15min":1.0009999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.274,"5min":1.369,"15min":1.369},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.998,"5min":1.0109999999999999,"15min":1.02},"min":{"1min":0.80600000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2609999999999999,"5min":1.409,"15min":1.409},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.67500000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1419999999999999,"5min":1.302,"15min":1.302},"last":1.083},{"interface":"front","average":{"1min":1.032,"5min":1.022,"15min":1.026},"min":{"1min":0.83599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.26,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.95699999999999996,"15min":0.96599999999999997},"min":{"1min":0.63400000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.147,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.105,"15min":1.131},"min":{"1min":0.73299999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.306,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":1.2230000000000001}]},{"osd":5,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.876,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.234,"5min":1.345,"15min":1.345},"last":1.129},{"interface":"front","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.161},"min":{"1min":0.999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.3240000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.111}]},{"osd":7,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1970000000000001,"15min":1.232},"min":{"1min":0.97399999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.377,"5min":1.631,"15min":1.631},"last":1.28},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.159,"15min":1.171},"min":{"1min":1,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4770000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.1950000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":471040,"data_stored":193537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2310000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.95199999999999996,"15min":0.96699999999999997},"min":{"1min":0.65300000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2749999999999999,"5min":1.371,"15min":1.371},"last":0.93100000000000005}]},{"osd":1,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.96699999999999997,"15min":0.98299999999999998},"min":{"1min":0.64300000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1060000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.98699999999999999,"15min":1.0309999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.167,"5min":1.708,"15min":1.708},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.998,"15min":1.03},"min":{"1min":0.72799999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2629999999999999,"5min":1.514,"15min":1.514},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0429999999999999,"15min":1.0669999999999999},"min":{"1min":0.73099999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.2090000000000001,"5min":1.476,"15min":1.476},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.1020000000000001,"15min":1.1399999999999999},"min":{"1min":0.82099999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.335,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":1.091,"15min":1.135},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3149999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.81299999999999994}]},{"osd":5,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.012,"15min":1.054},"min":{"1min":0.68500000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.208,"5min":1.419,"15min":1.419},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.137,"15min":1.1990000000000001},"min":{"1min":0.69699999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.3120000000000001,"5min":1.621,"15min":1.621},"last":1.087}]},{"osd":6,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.121,"15min":1.167},"min":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3240000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.198,"15min":1.264},"min":{"1min":0.82899999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.2629999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.0620000000000001}]},{"osd":7,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2789999999999999,"15min":1.353},"min":{"1min":0.86799999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3799999999999999,"5min":1.772,"15min":1.772},"last":1.139},{"interface":"front","average":{"1min":1.171,"5min":1.2989999999999999,"15min":1.345},"min":{"1min":0.91700000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.391,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0229999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301980,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069860,"statfs":{"total":96636764160,"available":96327536640,"internally_reserved":0,"allocated":475136,"data_stored":198256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.71099999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.248,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.04,"15min":1.042},"min":{"1min":0.75600000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2470000000000001,"5min":1.383,"15min":1.383},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.66700000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96499999999999997,"15min":0.95699999999999996},"min":{"1min":0.71699999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2070000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95399999999999996,"15min":0.96099999999999997},"min":{"1min":0.76700000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.177,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.996,"15min":1.012},"min":{"1min":0.69699999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.2949999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.016}]},{"osd":3,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.92400000000000004,"15min":0.92500000000000004},"min":{"1min":0.73399999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.01,"15min":0.98799999999999999},"min":{"1min":0.80900000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.96999999999999997}]},{"osd":4,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.036,"5min":1.0189999999999999,"15min":1.0249999999999999},"min":{"1min":0.73099999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.99199999999999999,"15min":0.99399999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1950000000000001,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.117,"15min":1.1080000000000001},"min":{"1min":0.997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.075}]},{"osd":7,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.071,"15min":1.075},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.198,"5min":1.1679999999999999,"15min":1.165},"min":{"1min":0.997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.3420000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.101}]}]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.998,"15min":0.998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.99299999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.879}]},{"osd":1,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.53000000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.03,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.69599999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.70699999999999996}]},{"osd":2,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.872,"15min":0.88200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.214,"5min":1.74,"15min":1.74},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.97899999999999998,"15min":0.99299999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.089,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.1659999999999999}]},{"osd":4,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1100000000000001,"15min":1.1319999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.444,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.252},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.022,"15min":1.0309999999999999},"min":{"1min":0.83599999999999997,"5min":0.755,"15min":0.755},"max":{"1min":1.4239999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.214}]},{"osd":5,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.167,"15min":1.181},"min":{"1min":0.76700000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5649999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.411},{"interface":"front","average":{"1min":1.028,"5min":1.1220000000000001,"15min":1.1699999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.448,"5min":1.581,"15min":1.581},"last":1.325}]},{"osd":6,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1819999999999999,"15min":1.204},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5529999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.296},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6160000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.5009999999999999}]},{"osd":7,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2030000000000001,"15min":1.2250000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5589999999999999,"5min":2.073,"15min":2.073},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2250000000000001,"15min":1.244},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.4890000000000001,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.3520000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302016,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069824,"statfs":{"total":96636764160,"available":96327499776,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.105,"5min":1.204,"15min":1.204},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.61099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0589999999999999,"5min":1.111,"15min":1.111},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.626,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1140000000000001,"5min":1.127,"15min":1.127},"last":1.038},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2270000000000001,"5min":1.278,"15min":1.278},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.57899999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0009999999999999,"5min":1.073,"15min":1.073},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.93000000000000005},"min":{"1min":0.73699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2090000000000001,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.117}]},{"osd":4,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.0089999999999999,"15min":0.996},"min":{"1min":0.77500000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.296,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.083,"15min":1.0900000000000001},"min":{"1min":0.82599999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.335,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.1359999999999999,"15min":1.155},"min":{"1min":0.85499999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.292,"5min":1.573,"15min":1.573},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.091,"5min":1.099,"15min":1.107},"min":{"1min":0.87,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3300000000000001,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.137,"15min":1.159},"min":{"1min":0.81399999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.3480000000000001,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.141,"5min":1.1739999999999999,"15min":1.1830000000000001},"min":{"1min":0.84499999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.399,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.238}]},{"osd":7,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2110000000000001,"15min":1.218},"min":{"1min":0.96499999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.202,"5min":1.2290000000000001,"15min":1.244},"min":{"1min":1.0129999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4259999999999999,"5min":1.861,"15min":1.861},"last":1.073}]}]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.52300000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.83699999999999997}]},{"osd":2,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.443,"15min":0.443},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.749},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90600000000000003,"15min":0.89100000000000001},"min":{"1min":0.70699999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":1.155},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90100000000000002,"15min":0.92400000000000004},"min":{"1min":0.66300000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0700000000000001,"5min":1.778,"15min":1.778},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.069,"15min":1.093},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4199999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0529999999999999,"15min":1.056},"min":{"1min":0.79200000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.4750000000000001,"5min":1.825,"15min":1.825},"last":1.52}]},{"osd":5,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.089,"15min":1.1040000000000001},"min":{"1min":0.879,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.2909999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.323},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.143,"15min":1.1559999999999999},"min":{"1min":0.85099999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.409,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.2310000000000001}]},{"osd":6,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1339999999999999,"15min":1.1319999999999999},"min":{"1min":0.90800000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.425,"5min":1.734,"15min":1.734},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1639999999999999,"15min":1.1599999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.478,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.4970000000000001}]},{"osd":7,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.1970000000000001,"15min":1.1930000000000001},"min":{"1min":1.0169999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.5920000000000001,"5min":1.861,"15min":1.861},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.95199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.7210000000000001,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.028}]}]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302592,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069248,"statfs":{"total":96636764160,"available":96326909952,"internally_reserved":0,"allocated":1097728,"data_stored":800325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66100000000000003,"15min":0.67900000000000005},"min":{"1min":0.497,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82199999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.72399999999999998}]},{"osd":2,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.53500000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.2310000000000001,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":0.90700000000000003,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83399999999999996,"15min":0.82999999999999996},"min":{"1min":0.66000000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.90800000000000003,"15min":0.90400000000000003},"min":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.333,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.056,"15min":1.044},"min":{"1min":0.80000000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3899999999999999,"5min":1.919,"15min":1.919},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.006,"15min":1.008},"min":{"1min":0.79800000000000004,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.0349999999999999}]},{"osd":5,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.022,"15min":1.0169999999999999},"min":{"1min":0.82299999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.1739999999999999,"15min":1.1990000000000001},"min":{"1min":0.84299999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4399999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.194,"15min":1.202},"min":{"1min":0.89600000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1619999999999999,"15min":1.1539999999999999},"min":{"1min":0.95199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.502,"5min":1.841,"15min":1.841},"last":1.115}]},{"osd":7,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1459999999999999,"15min":1.1359999999999999},"min":{"1min":0.90600000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2509999999999999,"15min":1.2529999999999999},"min":{"1min":1.0369999999999999,"5min":0.878,"15min":0.878},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.379}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T18:01:53.849 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-15T18:01:53.850 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T18:01:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0f5697700 1 Processor -- start 2024-08-15T18:01:53.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0f5697700 1 -- start start 2024-08-15T18:01:53.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0f5697700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff0f015d540 0x7ff0f00af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0f5697700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff0f006e830 0x7ff0f00a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0f5697700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0f00a9260 0x7ff0f00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0f5697700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f00b0270 con 0x7ff0f015d540 2024-08-15T18:01:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0f5697700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f00af660 con 0x7ff0f006e830 2024-08-15T18:01:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0f5697700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f00af940 con 0x7ff0f00a9260 2024-08-15T18:01:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0eeffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff0f015d540 0x7ff0f00af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0ee7fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff0f006e830 0x7ff0f00a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0eeffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff0f015d540 0x7ff0f00af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48432/0 (socket says 172.21.15.38:48432) 2024-08-15T18:01:53.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.954+0000 7ff0eeffd700 1 -- 172.21.15.38:0/2227263026 learned_addr learned my addr 172.21.15.38:0/2227263026 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:53.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0ef7fe700 1 --2- 172.21.15.38:0/2227263026 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0f00a9260 0x7ff0f00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0eeffd700 1 -- 172.21.15.38:0/2227263026 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0f00a9260 msgr2=0x7ff0f00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:53.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0eeffd700 1 --2- 172.21.15.38:0/2227263026 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0f00a9260 0x7ff0f00b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0eeffd700 1 -- 172.21.15.38:0/2227263026 >> v2:172.21.15.104:3300/0 conn(0x7ff0f006e830 msgr2=0x7ff0f00a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:53.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0eeffd700 1 --2- 172.21.15.38:0/2227263026 >> v2:172.21.15.104:3300/0 conn(0x7ff0f006e830 0x7ff0f00a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0eeffd700 1 -- 172.21.15.38:0/2227263026 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f00afc20 con 0x7ff0f015d540 2024-08-15T18:01:53.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0ee7fc700 1 --2- 172.21.15.38:0/2227263026 >> v2:172.21.15.104:3300/0 conn(0x7ff0f006e830 0x7ff0f00a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:01:53.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0eeffd700 1 --2- 172.21.15.38:0/2227263026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0f015d540 0x7ff0f00af030 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff0f006c970 tx=0x7ff0e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8696015b2a104ae6 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0edffb700 1 -- 172.21.15.38:0/2227263026 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e400b640 con 0x7ff0f015d540 2024-08-15T18:01:53.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0edffb700 1 -- 172.21.15.38:0/2227263026 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e400b7a0 con 0x7ff0f015d540 2024-08-15T18:01:53.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0edffb700 1 -- 172.21.15.38:0/2227263026 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e400bb60 con 0x7ff0f015d540 2024-08-15T18:01:53.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0f5697700 1 -- 172.21.15.38:0/2227263026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0f015d540 msgr2=0x7ff0f00af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0f5697700 1 --2- 172.21.15.38:0/2227263026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0f015d540 0x7ff0f00af030 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff0f006c970 tx=0x7ff0e4009250 comp rx=0 tx=0).stop 2024-08-15T18:01:53.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0f5697700 1 -- 172.21.15.38:0/2227263026 shutdown_connections 2024-08-15T18:01:53.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0f5697700 1 --2- 172.21.15.38:0/2227263026 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0f00a9260 0x7ff0f00b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0f5697700 1 --2- 172.21.15.38:0/2227263026 >> v2:172.21.15.104:3300/0 conn(0x7ff0f006e830 0x7ff0f00a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0f5697700 1 --2- 172.21.15.38:0/2227263026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0f015d540 0x7ff0f00af030 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0f5697700 1 -- 172.21.15.38:0/2227263026 >> 172.21.15.38:0/2227263026 conn(0x7ff0f0066e00 msgr2=0x7ff0f0067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:53.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0f5697700 1 -- 172.21.15.38:0/2227263026 shutdown_connections 2024-08-15T18:01:53.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.958+0000 7ff0f5697700 1 -- 172.21.15.38:0/2227263026 wait complete. 2024-08-15T18:01:53.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0f5697700 1 Processor -- start 2024-08-15T18:01:53.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0f5697700 1 -- start start 2024-08-15T18:01:53.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0f5697700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0f006e830 0x7ff0f0157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0f5697700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0f00a9260 0x7ff0f016d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0f5697700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0f015d540 0x7ff0f016df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0f5697700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f00c8740 con 0x7ff0f006e830 2024-08-15T18:01:53.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0f5697700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f00ad000 con 0x7ff0f015d540 2024-08-15T18:01:53.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0f5697700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f00ad2e0 con 0x7ff0f00a9260 2024-08-15T18:01:53.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0eeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0f006e830 0x7ff0f0157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0eeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0f006e830 0x7ff0f0157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48436/0 (socket says 172.21.15.38:48436) 2024-08-15T18:01:53.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0eeffd700 1 -- 172.21.15.38:0/2377261619 learned_addr learned my addr 172.21.15.38:0/2377261619 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:53.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0ef7fe700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0f015d540 0x7ff0f016df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0ee7fc700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0f00a9260 0x7ff0f016d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0eeffd700 1 -- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0f00a9260 msgr2=0x7ff0f016d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0eeffd700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0f00a9260 0x7ff0f016d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0eeffd700 1 -- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0f015d540 msgr2=0x7ff0f016df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:53.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0eeffd700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0f015d540 0x7ff0f016df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:53.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0eeffd700 1 -- 172.21.15.38:0/2377261619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e0009a20 con 0x7ff0f006e830 2024-08-15T18:01:53.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0eeffd700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0f006e830 0x7ff0f0157780 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff0f006f870 tx=0x7ff0e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0d6ffd700 1 -- 172.21.15.38:0/2377261619 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e400b910 con 0x7ff0f006e830 2024-08-15T18:01:53.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.962+0000 7ff0d6ffd700 1 -- 172.21.15.38:0/2377261619 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e400ba70 con 0x7ff0f006e830 2024-08-15T18:01:53.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.966+0000 7ff0f5697700 1 -- 172.21.15.38:0/2377261619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e40094f0 con 0x7ff0f006e830 2024-08-15T18:01:53.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.966+0000 7ff0d6ffd700 1 -- 172.21.15.38:0/2377261619 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4011630 con 0x7ff0f006e830 2024-08-15T18:01:53.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.966+0000 7ff0f5697700 1 -- 172.21.15.38:0/2377261619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f00ad8a0 con 0x7ff0f006e830 2024-08-15T18:01:53.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.966+0000 7ff0d6ffd700 1 -- 172.21.15.38:0/2377261619 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff0e40119a0 con 0x7ff0f006e830 2024-08-15T18:01:53.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.966+0000 7ff0f5697700 1 -- 172.21.15.38:0/2377261619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f00645c0 con 0x7ff0f006e830 2024-08-15T18:01:53.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.966+0000 7ff0d6ffd700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff0dc060890 0x7ff0dc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:53.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.966+0000 7ff0d6ffd700 1 -- 172.21.15.38:0/2377261619 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7ff0e407de90 con 0x7ff0f006e830 2024-08-15T18:01:53.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.966+0000 7ff0ee7fc700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff0dc060890 0x7ff0dc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:53.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.970+0000 7ff0ee7fc700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff0dc060890 0x7ff0dc062d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009fd0 tx=0x7ff0e000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:53.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:53.978+0000 7ff0d6ffd700 1 -- 172.21.15.38:0/2377261619 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff0e404e580 con 0x7ff0f006e830 2024-08-15T18:01:54.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.134+0000 7ff0f5697700 1 -- 172.21.15.38:0/2377261619 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff0f00c8880 con 0x7ff0dc060890 2024-08-15T18:01:54.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0d6ffd700 1 -- 172.21.15.38:0/2377261619 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48877 (secure 0 0 0) 0x7ff0e000f040 con 0x7ff0dc060890 2024-08-15T18:01:54.144 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:54.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 -- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff0dc060890 msgr2=0x7ff0dc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:54.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff0dc060890 0x7ff0dc062d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009fd0 tx=0x7ff0e000b040 comp rx=0 tx=0).stop 2024-08-15T18:01:54.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 -- 172.21.15.38:0/2377261619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0f006e830 msgr2=0x7ff0f0157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:54.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0f006e830 0x7ff0f0157780 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff0f006f870 tx=0x7ff0e4017000 comp rx=0 tx=0).stop 2024-08-15T18:01:54.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 -- 172.21.15.38:0/2377261619 shutdown_connections 2024-08-15T18:01:54.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff0dc060890 0x7ff0dc062d40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0f015d540 0x7ff0f016df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0f00a9260 0x7ff0f016d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 --2- 172.21.15.38:0/2377261619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0f006e830 0x7ff0f0157780 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 -- 172.21.15.38:0/2377261619 >> 172.21.15.38:0/2377261619 conn(0x7ff0f0066e00 msgr2=0x7ff0f015c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:54.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 -- 172.21.15.38:0/2377261619 shutdown_connections 2024-08-15T18:01:54.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.142+0000 7ff0f5697700 1 -- 172.21.15.38:0/2377261619 wait complete. 2024-08-15T18:01:54.148 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-08-15T18:01:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-08-15T18:01:52.246011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":2408792,"kb_used_data":4960,"kb_used_omap":13,"kb_used_meta":2403698,"kb_avail":752565928,"statfs":{"total":773094113280,"available":770627510272,"internally_reserved":0,"allocated":5079040,"data_stored":2797077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2461387456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001682"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-08-15T18:01:44.112384+0000","last_change":"2024-08-15T17:58:24.394014+0000","last_active":"2024-08-15T18:01:44.112384+0000","last_peered":"2024-08-15T18:01:44.112384+0000","last_clean":"2024-08-15T18:01:44.112384+0000","last_became_active":"2024-08-15T17:58:24.391735+0000","last_became_peered":"2024-08-15T17:58:24.391735+0000","last_unstale":"2024-08-15T18:01:44.112384+0000","last_undegraded":"2024-08-15T18:01:44.112384+0000","last_fullsized":"2024-08-15T18:01:44.112384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_clean_scrub_stamp":"2024-08-15T17:58:22.996335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:31.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302552,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069288,"statfs":{"total":96636764160,"available":96326950912,"internally_reserved":0,"allocated":1056768,"data_stored":782443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92600000000000005,"15min":0.92800000000000005},"min":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1919999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.86299999999999999,"15min":0.871},"min":{"1min":0.64300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.208,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0589999999999999}]},{"osd":1,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91800000000000004,"15min":0.90700000000000003},"min":{"1min":0.67300000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89200000000000002,"15min":0.875},"min":{"1min":0.76900000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0880000000000001,"5min":1.165,"15min":1.165},"last":0.746}]},{"osd":2,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.73099999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.262,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.155},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1,"15min":1.022},"min":{"1min":0.65400000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.111,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.94599999999999995,"15min":0.93999999999999995},"min":{"1min":0.73799999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.01},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0880000000000001,"15min":1.083},"min":{"1min":0.86199999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3160000000000001,"5min":1.415,"15min":1.415},"last":1.26}]},{"osd":4,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.93999999999999995,"15min":0.96199999999999997},"min":{"1min":0.622,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.21,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.034,"5min":1.0529999999999999,"15min":1.0780000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2350000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.329}]},{"osd":5,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.105,"15min":1.1319999999999999},"min":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3080000000000001,"5min":1.599,"15min":1.599},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.131,"15min":1.1519999999999999},"min":{"1min":0.78400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3660000000000001,"5min":1.486,"15min":1.486},"last":0.98399999999999999}]},{"osd":6,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1970000000000001,"15min":1.2170000000000001},"min":{"1min":0.85299999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.379,"5min":1.631,"15min":1.631},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1850000000000001,"15min":1.1950000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.49,"5min":1.518,"15min":1.518},"last":1.498}]}]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":466944,"data_stored":192645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85199999999999998,"15min":0.81999999999999995},"min":{"1min":0.58999999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.224,"5min":1.226,"15min":1.226},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.91400000000000003,"15min":0.93600000000000005},"min":{"1min":0.622,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1970000000000001,"5min":1.516,"15min":1.516},"last":1.155}]},{"osd":1,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88600000000000001,"15min":0.90800000000000003},"min":{"1min":0.66000000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.054,"5min":1.333,"15min":1.333},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.99299999999999999,"15min":1.006},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2669999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.84799999999999998}]},{"osd":2,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.98299999999999998,"15min":1.0009999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.274,"5min":1.369,"15min":1.369},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.998,"5min":1.0109999999999999,"15min":1.02},"min":{"1min":0.80600000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2609999999999999,"5min":1.409,"15min":1.409},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.67500000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1419999999999999,"5min":1.302,"15min":1.302},"last":1.083},{"interface":"front","average":{"1min":1.032,"5min":1.022,"15min":1.026},"min":{"1min":0.83599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.26,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.95699999999999996,"15min":0.96599999999999997},"min":{"1min":0.63400000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.147,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.105,"15min":1.131},"min":{"1min":0.73299999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.306,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":1.2230000000000001}]},{"osd":5,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.876,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.234,"5min":1.345,"15min":1.345},"last":1.129},{"interface":"front","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.161},"min":{"1min":0.999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.3240000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.111}]},{"osd":7,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1970000000000001,"15min":1.232},"min":{"1min":0.97399999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.377,"5min":1.631,"15min":1.631},"last":1.28},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.159,"15min":1.171},"min":{"1min":1,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4770000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.1950000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":471040,"data_stored":193537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2310000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.95199999999999996,"15min":0.96699999999999997},"min":{"1min":0.65300000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2749999999999999,"5min":1.371,"15min":1.371},"last":0.93100000000000005}]},{"osd":1,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.96699999999999997,"15min":0.98299999999999998},"min":{"1min":0.64300000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1060000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.98699999999999999,"15min":1.0309999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.167,"5min":1.708,"15min":1.708},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.998,"15min":1.03},"min":{"1min":0.72799999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2629999999999999,"5min":1.514,"15min":1.514},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0429999999999999,"15min":1.0669999999999999},"min":{"1min":0.73099999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.2090000000000001,"5min":1.476,"15min":1.476},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.1020000000000001,"15min":1.1399999999999999},"min":{"1min":0.82099999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.335,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":1.091,"15min":1.135},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3149999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.81299999999999994}]},{"osd":5,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.012,"15min":1.054},"min":{"1min":0.68500000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.208,"5min":1.419,"15min":1.419},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.137,"15min":1.1990000000000001},"min":{"1min":0.69699999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.3120000000000001,"5min":1.621,"15min":1.621},"last":1.087}]},{"osd":6,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.121,"15min":1.167},"min":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3240000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.198,"15min":1.264},"min":{"1min":0.82899999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.2629999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.0620000000000001}]},{"osd":7,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2789999999999999,"15min":1.353},"min":{"1min":0.86799999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3799999999999999,"5min":1.772,"15min":1.772},"last":1.139},{"interface":"front","average":{"1min":1.171,"5min":1.2989999999999999,"15min":1.345},"min":{"1min":0.91700000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.391,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0229999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301980,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069860,"statfs":{"total":96636764160,"available":96327536640,"internally_reserved":0,"allocated":475136,"data_stored":198256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.71099999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.248,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.04,"15min":1.042},"min":{"1min":0.75600000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2470000000000001,"5min":1.383,"15min":1.383},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.66700000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96499999999999997,"15min":0.95699999999999996},"min":{"1min":0.71699999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2070000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95399999999999996,"15min":0.96099999999999997},"min":{"1min":0.76700000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.177,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.996,"15min":1.012},"min":{"1min":0.69699999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.2949999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.016}]},{"osd":3,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.92400000000000004,"15min":0.92500000000000004},"min":{"1min":0.73399999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.01,"15min":0.98799999999999999},"min":{"1min":0.80900000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.96999999999999997}]},{"osd":4,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.036,"5min":1.0189999999999999,"15min":1.0249999999999999},"min":{"1min":0.73099999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.99199999999999999,"15min":0.99399999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1950000000000001,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.117,"15min":1.1080000000000001},"min":{"1min":0.997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.075}]},{"osd":7,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.071,"15min":1.075},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.198,"5min":1.1679999999999999,"15min":1.165},"min":{"1min":0.997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.3420000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.101}]}]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.998,"15min":0.998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.99299999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.879}]},{"osd":1,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.53000000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.03,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.69599999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.70699999999999996}]},{"osd":2,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.872,"15min":0.88200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.214,"5min":1.74,"15min":1.74},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.97899999999999998,"15min":0.99299999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.089,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.1659999999999999}]},{"osd":4,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1100000000000001,"15min":1.1319999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.444,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.252},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.022,"15min":1.0309999999999999},"min":{"1min":0.83599999999999997,"5min":0.755,"15min":0.755},"max":{"1min":1.4239999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.214}]},{"osd":5,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.167,"15min":1.181},"min":{"1min":0.76700000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5649999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.411},{"interface":"front","average":{"1min":1.028,"5min":1.1220000000000001,"15min":1.1699999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.448,"5min":1.581,"15min":1.581},"last":1.325}]},{"osd":6,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1819999999999999,"15min":1.204},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5529999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.296},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6160000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.5009999999999999}]},{"osd":7,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2030000000000001,"15min":1.2250000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5589999999999999,"5min":2.073,"15min":2.073},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2250000000000001,"15min":1.244},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.4890000000000001,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.3520000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302016,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069824,"statfs":{"total":96636764160,"available":96327499776,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.105,"5min":1.204,"15min":1.204},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.61099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0589999999999999,"5min":1.111,"15min":1.111},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.626,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1140000000000001,"5min":1.127,"15min":1.127},"last":1.038},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2270000000000001,"5min":1.278,"15min":1.278},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.57899999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0009999999999999,"5min":1.073,"15min":1.073},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.93000000000000005},"min":{"1min":0.73699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2090000000000001,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.117}]},{"osd":4,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.0089999999999999,"15min":0.996},"min":{"1min":0.77500000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.296,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.083,"15min":1.0900000000000001},"min":{"1min":0.82599999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.335,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.1359999999999999,"15min":1.155},"min":{"1min":0.85499999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.292,"5min":1.573,"15min":1.573},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.091,"5min":1.099,"15min":1.107},"min":{"1min":0.87,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3300000000000001,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.137,"15min":1.159},"min":{"1min":0.81399999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.3480000000000001,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.141,"5min":1.1739999999999999,"15min":1.1830000000000001},"min":{"1min":0.84499999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.399,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.238}]},{"osd":7,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2110000000000001,"15min":1.218},"min":{"1min":0.96499999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.202,"5min":1.2290000000000001,"15min":1.244},"min":{"1min":1.0129999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4259999999999999,"5min":1.861,"15min":1.861},"last":1.073}]}]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.52300000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.83699999999999997}]},{"osd":2,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.443,"15min":0.443},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.749},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90600000000000003,"15min":0.89100000000000001},"min":{"1min":0.70699999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":1.155},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90100000000000002,"15min":0.92400000000000004},"min":{"1min":0.66300000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0700000000000001,"5min":1.778,"15min":1.778},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.069,"15min":1.093},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4199999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0529999999999999,"15min":1.056},"min":{"1min":0.79200000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.4750000000000001,"5min":1.825,"15min":1.825},"last":1.52}]},{"osd":5,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.089,"15min":1.1040000000000001},"min":{"1min":0.879,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.2909999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.323},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.143,"15min":1.1559999999999999},"min":{"1min":0.85099999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.409,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.2310000000000001}]},{"osd":6,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1339999999999999,"15min":1.1319999999999999},"min":{"1min":0.90800000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.425,"5min":1.734,"15min":1.734},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1639999999999999,"15min":1.1599999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.478,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.4970000000000001}]},{"osd":7,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.1970000000000001,"15min":1.1930000000000001},"min":{"1min":1.0169999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.5920000000000001,"5min":1.861,"15min":1.861},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.95199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.7210000000000001,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.028}]}]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302592,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069248,"statfs":{"total":96636764160,"available":96326909952,"internally_reserved":0,"allocated":1097728,"data_stored":800325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66100000000000003,"15min":0.67900000000000005},"min":{"1min":0.497,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82199999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.72399999999999998}]},{"osd":2,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.53500000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.2310000000000001,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":0.90700000000000003,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83399999999999996,"15min":0.82999999999999996},"min":{"1min":0.66000000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.90800000000000003,"15min":0.90400000000000003},"min":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.333,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.056,"15min":1.044},"min":{"1min":0.80000000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3899999999999999,"5min":1.919,"15min":1.919},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.006,"15min":1.008},"min":{"1min":0.79800000000000004,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.0349999999999999}]},{"osd":5,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.022,"15min":1.0169999999999999},"min":{"1min":0.82299999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.1739999999999999,"15min":1.1990000000000001},"min":{"1min":0.84299999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4399999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.194,"15min":1.202},"min":{"1min":0.89600000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1619999999999999,"15min":1.1539999999999999},"min":{"1min":0.95199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.502,"5min":1.841,"15min":1.841},"last":1.115}]},{"osd":7,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1459999999999999,"15min":1.1359999999999999},"min":{"1min":0.90600000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2509999999999999,"15min":1.2529999999999999},"min":{"1min":1.0369999999999999,"5min":0.878,"15min":0.878},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.379}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T18:01:54.161 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T18:01:54.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.270+0000 7f5e5cde0700 1 Processor -- start 2024-08-15T18:01:54.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.270+0000 7f5e5cde0700 1 -- start start 2024-08-15T18:01:54.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.270+0000 7f5e5cde0700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5e581613e0 0x7f5e581617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:54.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.270+0000 7f5e5cde0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e580b1a90 0x7f5e580ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:54.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.270+0000 7f5e5cde0700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e580adfa0 0x7f5e580ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:54.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.270+0000 7f5e5cde0700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e580acf00 con 0x7f5e580b1a90 2024-08-15T18:01:54.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.270+0000 7f5e5cde0700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e580ad1e0 con 0x7f5e581613e0 2024-08-15T18:01:54.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.270+0000 7f5e5cde0700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580ad4c0 con 0x7f5e580adfa0 2024-08-15T18:01:54.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e56ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e580b1a90 0x7f5e580ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:54.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e56ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e580b1a90 0x7f5e580ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48440/0 (socket says 172.21.15.38:48440) 2024-08-15T18:01:54.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e56ffd700 1 -- 172.21.15.38:0/1783823254 learned_addr learned my addr 172.21.15.38:0/1783823254 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:54.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e577fe700 1 --2- 172.21.15.38:0/1783823254 >> v2:172.21.15.104:3300/0 conn(0x7f5e581613e0 0x7f5e581617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:54.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e57fff700 1 --2- 172.21.15.38:0/1783823254 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e580adfa0 0x7f5e580ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:54.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e56ffd700 1 -- 172.21.15.38:0/1783823254 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e580adfa0 msgr2=0x7f5e580ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:54.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e56ffd700 1 --2- 172.21.15.38:0/1783823254 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e580adfa0 0x7f5e580ae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e56ffd700 1 -- 172.21.15.38:0/1783823254 >> v2:172.21.15.104:3300/0 conn(0x7f5e581613e0 msgr2=0x7f5e581617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:54.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e56ffd700 1 --2- 172.21.15.38:0/1783823254 >> v2:172.21.15.104:3300/0 conn(0x7f5e581613e0 0x7f5e581617b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e56ffd700 1 -- 172.21.15.38:0/1783823254 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e580ad7a0 con 0x7f5e580b1a90 2024-08-15T18:01:54.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e56ffd700 1 --2- 172.21.15.38:0/1783823254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e580b1a90 0x7f5e580ac6d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e58070260 tx=0x7f5e40009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905ff9b4d8882cf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e567fc700 1 -- 172.21.15.38:0/1783823254 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000e070 con 0x7f5e580b1a90 2024-08-15T18:01:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e567fc700 1 -- 172.21.15.38:0/1783823254 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4000ba20 con 0x7f5e580b1a90 2024-08-15T18:01:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e567fc700 1 -- 172.21.15.38:0/1783823254 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e400092e0 con 0x7f5e580b1a90 2024-08-15T18:01:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e5cde0700 1 -- 172.21.15.38:0/1783823254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e580b1a90 msgr2=0x7f5e580ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e5cde0700 1 --2- 172.21.15.38:0/1783823254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e580b1a90 0x7f5e580ac6d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e58070260 tx=0x7f5e40009840 comp rx=0 tx=0).stop 2024-08-15T18:01:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e5cde0700 1 -- 172.21.15.38:0/1783823254 shutdown_connections 2024-08-15T18:01:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e5cde0700 1 --2- 172.21.15.38:0/1783823254 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e580adfa0 0x7f5e580ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e5cde0700 1 --2- 172.21.15.38:0/1783823254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e580b1a90 0x7f5e580ac6d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e5cde0700 1 --2- 172.21.15.38:0/1783823254 >> v2:172.21.15.104:3300/0 conn(0x7f5e581613e0 0x7f5e581617b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e5cde0700 1 -- 172.21.15.38:0/1783823254 >> 172.21.15.38:0/1783823254 conn(0x7f5e580c9840 msgr2=0x7f5e580c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:54.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e5cde0700 1 -- 172.21.15.38:0/1783823254 shutdown_connections 2024-08-15T18:01:54.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.274+0000 7f5e5cde0700 1 -- 172.21.15.38:0/1783823254 wait complete. 2024-08-15T18:01:54.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e5cde0700 1 Processor -- start 2024-08-15T18:01:54.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e5cde0700 1 -- start start 2024-08-15T18:01:54.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e5cde0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e580adfa0 0x7f5e5814b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:54.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e5cde0700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e580b1a90 0x7f5e580c2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e5cde0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e581613e0 0x7f5e580c2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e5cde0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e58069c40 con 0x7f5e580adfa0 2024-08-15T18:01:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e5cde0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5814b540 con 0x7f5e581613e0 2024-08-15T18:01:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e5cde0700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e58069e90 con 0x7f5e580b1a90 2024-08-15T18:01:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e577fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e580adfa0 0x7f5e5814b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e577fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e580adfa0 0x7f5e5814b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48456/0 (socket says 172.21.15.38:48456) 2024-08-15T18:01:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e577fe700 1 -- 172.21.15.38:0/30476423 learned_addr learned my addr 172.21.15.38:0/30476423 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:54.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e57fff700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e581613e0 0x7f5e580c2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e56ffd700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e580b1a90 0x7f5e580c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e577fe700 1 -- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e580b1a90 msgr2=0x7f5e580c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e577fe700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e580b1a90 0x7f5e580c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e577fe700 1 -- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e581613e0 msgr2=0x7f5e580c2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e577fe700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e581613e0 0x7f5e580c2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e577fe700 1 -- 172.21.15.38:0/30476423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e40009ae0 con 0x7f5e580adfa0 2024-08-15T18:01:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e57fff700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e581613e0 0x7f5e580c2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:01:54.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e577fe700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e580adfa0 0x7f5e5814b000 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800e930 tx=0x7f5e480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e4f7fe700 1 -- 172.21.15.38:0/30476423 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48013680 con 0x7f5e580adfa0 2024-08-15T18:01:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e4f7fe700 1 -- 172.21.15.38:0/30476423 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e480137e0 con 0x7f5e580adfa0 2024-08-15T18:01:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e5cde0700 1 -- 172.21.15.38:0/30476423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5806a170 con 0x7f5e580adfa0 2024-08-15T18:01:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e4f7fe700 1 -- 172.21.15.38:0/30476423 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48013b80 con 0x7f5e580adfa0 2024-08-15T18:01:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.278+0000 7f5e5cde0700 1 -- 172.21.15.38:0/30476423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5806a730 con 0x7f5e580adfa0 2024-08-15T18:01:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.282+0000 7f5e5cde0700 1 -- 172.21.15.38:0/30476423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5815a830 con 0x7f5e580adfa0 2024-08-15T18:01:54.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.282+0000 7f5e4f7fe700 1 -- 172.21.15.38:0/30476423 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5e4800c3e0 con 0x7f5e580adfa0 2024-08-15T18:01:54.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.282+0000 7f5e4f7fe700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e38060860 0x7f5e38062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:54.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.282+0000 7f5e4f7fe700 1 -- 172.21.15.38:0/30476423 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f5e4807cfe0 con 0x7f5e580adfa0 2024-08-15T18:01:54.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.282+0000 7f5e56ffd700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e38060860 0x7f5e38062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:54.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.282+0000 7f5e56ffd700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e38060860 0x7f5e38062d10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009880 tx=0x7f5e40018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.290+0000 7f5e4f7fe700 1 -- 172.21.15.38:0/30476423 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5e4804d750 con 0x7f5e580adfa0 2024-08-15T18:01:54.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.453+0000 7f5e5cde0700 1 -- 172.21.15.38:0/30476423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e5815a830 con 0x7f5e580adfa0 2024-08-15T18:01:54.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.453+0000 7f5e4f7fe700 1 -- 172.21.15.38:0/30476423 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12395 (secure 0 0 0) 0x7f5e5815a830 con 0x7f5e580adfa0 2024-08-15T18:01:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:54.457 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":43,"fsid":"3c800201-a1b3-4c6d-887f-37b16c41eb0e","created":"2024-08-15T17:58:11.936849+0000","modified":"2024-08-15T18:01:51.113831+0000","last_up_change":"2024-08-15T17:58:22.996335+0000","last_in_change":"2024-08-15T17:58:13.978121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:58:22.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d94c8b75-7d55-4eb9-b831-c22e47c948ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24121},{"type":"v1","addr":"172.21.15.38:6809","nonce":24121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24121},{"type":"v1","addr":"172.21.15.38:6811","nonce":24121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24121},{"type":"v1","addr":"172.21.15.38:6815","nonce":24121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":24121},{"type":"v1","addr":"172.21.15.38:6813","nonce":24121}]},"public_addr":"172.21.15.38:6809/24121","cluster_addr":"172.21.15.38:6811/24121","heartbeat_back_addr":"172.21.15.38:6815/24121","heartbeat_front_addr":"172.21.15.38:6813/24121","state":["exists","up"]},{"osd":1,"uuid":"bbe82cfa-6bc8-43ef-b606-7802ad47cf8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":24122},{"type":"v1","addr":"172.21.15.38:6817","nonce":24122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24122},{"type":"v1","addr":"172.21.15.38:6819","nonce":24122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24122},{"type":"v1","addr":"172.21.15.38:6823","nonce":24122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24122},{"type":"v1","addr":"172.21.15.38:6821","nonce":24122}]},"public_addr":"172.21.15.38:6817/24122","cluster_addr":"172.21.15.38:6819/24122","heartbeat_back_addr":"172.21.15.38:6823/24122","heartbeat_front_addr":"172.21.15.38:6821/24122","state":["exists","up"]},{"osd":2,"uuid":"c87912dd-fe9f-4142-9c92-3ae704a8d4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":24123},{"type":"v1","addr":"172.21.15.38:6825","nonce":24123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":24123},{"type":"v1","addr":"172.21.15.38:6827","nonce":24123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":24123},{"type":"v1","addr":"172.21.15.38:6831","nonce":24123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":24123},{"type":"v1","addr":"172.21.15.38:6829","nonce":24123}]},"public_addr":"172.21.15.38:6825/24123","cluster_addr":"172.21.15.38:6827/24123","heartbeat_back_addr":"172.21.15.38:6831/24123","heartbeat_front_addr":"172.21.15.38:6829/24123","state":["exists","up"]},{"osd":3,"uuid":"177bca56-b19b-4580-9bb4-483cabde4b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24120},{"type":"v1","addr":"172.21.15.38:6801","nonce":24120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24120},{"type":"v1","addr":"172.21.15.38:6803","nonce":24120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":24120},{"type":"v1","addr":"172.21.15.38:6807","nonce":24120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24120},{"type":"v1","addr":"172.21.15.38:6805","nonce":24120}]},"public_addr":"172.21.15.38:6801/24120","cluster_addr":"172.21.15.38:6803/24120","heartbeat_back_addr":"172.21.15.38:6807/24120","heartbeat_front_addr":"172.21.15.38:6805/24120","state":["exists","up"]},{"osd":4,"uuid":"959df71d-f02f-4b4f-a79a-dd9194728ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":24496},{"type":"v1","addr":"172.21.15.104:6825","nonce":24496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":24496},{"type":"v1","addr":"172.21.15.104:6827","nonce":24496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":24496},{"type":"v1","addr":"172.21.15.104:6831","nonce":24496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":24496},{"type":"v1","addr":"172.21.15.104:6829","nonce":24496}]},"public_addr":"172.21.15.104:6825/24496","cluster_addr":"172.21.15.104:6827/24496","heartbeat_back_addr":"172.21.15.104:6831/24496","heartbeat_front_addr":"172.21.15.104:6829/24496","state":["exists","up"]},{"osd":5,"uuid":"023ef8e6-1845-4938-afaa-eeb0ea8e135f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":24498},{"type":"v1","addr":"172.21.15.104:6801","nonce":24498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":24498},{"type":"v1","addr":"172.21.15.104:6803","nonce":24498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":24498},{"type":"v1","addr":"172.21.15.104:6807","nonce":24498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":24498},{"type":"v1","addr":"172.21.15.104:6805","nonce":24498}]},"public_addr":"172.21.15.104:6801/24498","cluster_addr":"172.21.15.104:6803/24498","heartbeat_back_addr":"172.21.15.104:6807/24498","heartbeat_front_addr":"172.21.15.104:6805/24498","state":["exists","up"]},{"osd":6,"uuid":"750271d8-321b-47bf-ad2e-2b5e6de7d983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":24497},{"type":"v1","addr":"172.21.15.104:6809","nonce":24497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":24497},{"type":"v1","addr":"172.21.15.104:6811","nonce":24497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":24497},{"type":"v1","addr":"172.21.15.104:6815","nonce":24497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":24497},{"type":"v1","addr":"172.21.15.104:6813","nonce":24497}]},"public_addr":"172.21.15.104:6809/24497","cluster_addr":"172.21.15.104:6811/24497","heartbeat_back_addr":"172.21.15.104:6815/24497","heartbeat_front_addr":"172.21.15.104:6813/24497","state":["exists","up"]},{"osd":7,"uuid":"ea9c9c4a-9662-4a09-b81f-260ccc5c27d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":24504},{"type":"v1","addr":"172.21.15.104:6817","nonce":24504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":24504},{"type":"v1","addr":"172.21.15.104:6819","nonce":24504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":24504},{"type":"v1","addr":"172.21.15.104:6823","nonce":24504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":24504},{"type":"v1","addr":"172.21.15.104:6821","nonce":24504}]},"public_addr":"172.21.15.104:6817/24504","cluster_addr":"172.21.15.104:6819/24504","heartbeat_back_addr":"172.21.15.104:6823/24504","heartbeat_front_addr":"172.21.15.104:6821/24504","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.052118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.373492+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.570417+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.135376+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.396500+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.154875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.291342+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:58:19.204684+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:6834/875828080":"2024-08-16T18:01:48.089417+0000","172.21.15.104:6837/1865762791":"2024-08-16T18:01:41.797233+0000","172.21.15.104:6835/875828080":"2024-08-16T18:01:48.089417+0000","172.21.15.104:6836/1865762791":"2024-08-16T18:01:41.797233+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T18:01:54.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 -- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e38060860 msgr2=0x7f5e38062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:54.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e38060860 0x7f5e38062d10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009880 tx=0x7f5e40018000 comp rx=0 tx=0).stop 2024-08-15T18:01:54.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 -- 172.21.15.38:0/30476423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e580adfa0 msgr2=0x7f5e5814b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:54.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e580adfa0 0x7f5e5814b000 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800e930 tx=0x7f5e480091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 -- 172.21.15.38:0/30476423 shutdown_connections 2024-08-15T18:01:54.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5e38060860 0x7f5e38062d10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e581613e0 0x7f5e580c2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e580b1a90 0x7f5e580c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 --2- 172.21.15.38:0/30476423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e580adfa0 0x7f5e5814b000 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:54.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 -- 172.21.15.38:0/30476423 >> 172.21.15.38:0/30476423 conn(0x7f5e580c9840 msgr2=0x7f5e580b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:54.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 -- 172.21.15.38:0/30476423 shutdown_connections 2024-08-15T18:01:54.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:54.457+0000 7f5e5cde0700 1 -- 172.21.15.38:0/30476423 wait complete. 2024-08-15T18:01:55.478 INFO:tasks.ceph:Scrubbing osd.0 2024-08-15T18:01:55.478 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-08-15T18:01:55.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.601+0000 7f4ec3fff700 1 Processor -- start 2024-08-15T18:01:55.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.601+0000 7f4ec3fff700 1 -- start start 2024-08-15T18:01:55.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.601+0000 7f4ec3fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f4ec4068db0 0x7f4ec406aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.601+0000 7f4ec3fff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec406af80 con 0x7f4ec4155c90 2024-08-15T18:01:55.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.601+0000 7f4ec3fff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec406ba50 con 0x7f4ec40af4d0 2024-08-15T18:01:55.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.601+0000 7f4ec3fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec40ac500 con 0x7f4ec4068db0 2024-08-15T18:01:55.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.601+0000 7f4ec2ffd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4ec4155c90 0x7f4ec40686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40426/0 (socket says 172.21.15.38:40426) 2024-08-15T18:01:55.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.601+0000 7f4ec2ffd700 1 -- 172.21.15.38:0/3366538828 learned_addr learned my addr 172.21.15.38:0/3366538828 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:55.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4eba7fc700 1 --2- 172.21.15.38:0/3366538828 >> v2:172.21.15.104:3300/0 conn(0x7f4ec4068db0 0x7f4ec406aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332270037 0 0) 0x7f4ec406af80 con 0x7f4ec4155c90 2024-08-15T18:01:55.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eb0003570 con 0x7f4ec4155c90 2024-08-15T18:01:55.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763625061 0 0) 0x7f4ec406ba50 con 0x7f4ec40af4d0 2024-08-15T18:01:55.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec406af80 con 0x7f4ec40af4d0 2024-08-15T18:01:55.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3165709307 0 0) 0x7f4eb0003570 con 0x7f4ec4155c90 2024-08-15T18:01:55.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 >> v2:172.21.15.104:3300/0 conn(0x7f4ec4068db0 msgr2=0x7f4ec406aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:55.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 --2- 172.21.15.38:0/3366538828 >> v2:172.21.15.104:3300/0 conn(0x7f4ec4068db0 0x7f4ec406aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 >> v1:172.21.15.104:6789/0 conn(0x7f4ec40af4d0 legacy=0x7f4ec40ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec406b1b0 con 0x7f4ec4155c90 2024-08-15T18:01:55.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f4eb4001470 con 0x7f4ec4155c90 2024-08-15T18:01:55.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4eba7fc700 1 --2- 172.21.15.38:0/3366538828 >> v2:172.21.15.104:3300/0 conn(0x7f4ec4068db0 0x7f4ec406aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:01:55.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 >> v1:172.21.15.38:6789/0 conn(0x7f4ec4155c90 legacy=0x7f4ec40686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4eb0003c40 0x7f4eb0004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4eb0004810 0x7f4eb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb0005210 0x7f4eb000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40ac500 con 0x7f4eb0005210 2024-08-15T18:01:55.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb000c1b0 con 0x7f4eb0004810 2024-08-15T18:01:55.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb000c370 con 0x7f4eb0003c40 2024-08-15T18:01:55.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec37fe700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb0005210 0x7f4eb000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4eba7fc700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4eb0004810 0x7f4eb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec2ffd700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4eb0003c40 0x7f4eb0004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec37fe700 1 -- 172.21.15.38:0/3366538828 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4eb0003c40 msgr2=0x7f4eb0004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:55.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec37fe700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4eb0003c40 0x7f4eb0004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec37fe700 1 -- 172.21.15.38:0/3366538828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4eb0004810 msgr2=0x7f4eb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:55.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec37fe700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4eb0004810 0x7f4eb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec37fe700 1 -- 172.21.15.38:0/3366538828 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb0003570 con 0x7f4eb0005210 2024-08-15T18:01:55.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec2ffd700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4eb0003c40 0x7f4eb0004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:55.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec37fe700 1 -- 172.21.15.38:0/3366538828 reap_dead start 2024-08-15T18:01:55.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec37fe700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb0005210 0x7f4eb000ba60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40af8f0 tx=0x7f4ebc0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8a4500df47d74b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:55.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc001ab0 con 0x7f4eb0005210 2024-08-15T18:01:55.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc006ba0 con 0x7f4eb0005210 2024-08-15T18:01:55.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec27fc700 1 -- 172.21.15.38:0/3366538828 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00c8b0 con 0x7f4eb0005210 2024-08-15T18:01:55.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3366538828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb0005210 msgr2=0x7f4eb000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec3fff700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb0005210 0x7f4eb000ba60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40af8f0 tx=0x7f4ebc0017a0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3366538828 shutdown_connections 2024-08-15T18:01:55.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec3fff700 1 --2- 172.21.15.38:0/3366538828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb0005210 0x7f4eb000ba60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.605+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3366538828 >> 172.21.15.38:0/3366538828 conn(0x7f4ec40ccfa0 msgr2=0x7f4ec406d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:55.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.609+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3366538828 shutdown_connections 2024-08-15T18:01:55.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.609+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3366538828 wait complete. 2024-08-15T18:01:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.609+0000 7f4ec3fff700 1 Processor -- start 2024-08-15T18:01:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.609+0000 7f4ec3fff700 1 -- start start 2024-08-15T18:01:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4eb0005210 0x7f4ec4153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec40c3180 0x7f4ec40c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec3fff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ec40c1450 0x7f4ec40c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec3fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40c3b30 con 0x7f4ec40c3180 2024-08-15T18:01:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec3fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40c2160 con 0x7f4eb0005210 2024-08-15T18:01:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec3fff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40c2440 con 0x7f4ec40c1450 2024-08-15T18:01:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4eba7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec40c3180 0x7f4ec40c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4eba7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec40c3180 0x7f4ec40c35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48478/0 (socket says 172.21.15.38:48478) 2024-08-15T18:01:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4eba7fc700 1 -- 172.21.15.38:0/3845301962 learned_addr learned my addr 172.21.15.38:0/3845301962 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec2ffd700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4eb0005210 0x7f4ec4153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec37fe700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ec40c1450 0x7f4ec40c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4eba7fc700 1 -- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ec40c1450 msgr2=0x7f4ec40c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4eba7fc700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ec40c1450 0x7f4ec40c18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4eba7fc700 1 -- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4eb0005210 msgr2=0x7f4ec4153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4eba7fc700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4eb0005210 0x7f4ec4153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4eba7fc700 1 -- 172.21.15.38:0/3845301962 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec40c2720 con 0x7f4ec40c3180 2024-08-15T18:01:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec2ffd700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4eb0005210 0x7f4ec4153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4eba7fc700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec40c3180 0x7f4ec40c35f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4eac011d70 tx=0x7f4eac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ebb7fe700 1 -- 172.21.15.38:0/3845301962 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac011e70 con 0x7f4ec40c3180 2024-08-15T18:01:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3845301962 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec40c2a00 con 0x7f4ec40c3180 2024-08-15T18:01:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ebb7fe700 1 -- 172.21.15.38:0/3845301962 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac00fcd0 con 0x7f4ec40c3180 2024-08-15T18:01:55.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ebb7fe700 1 -- 172.21.15.38:0/3845301962 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac01d6d0 con 0x7f4ec40c3180 2024-08-15T18:01:55.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3845301962 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec40c1e00 con 0x7f4ec40c3180 2024-08-15T18:01:55.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3845301962 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4ec40c8de0 con 0x7f4ec40c3180 2024-08-15T18:01:55.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.613+0000 7f4ebb7fe700 1 -- 172.21.15.38:0/3845301962 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4eac01dac0 con 0x7f4ec40c3180 2024-08-15T18:01:55.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.617+0000 7f4ebb7fe700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4ea4060860 0x7f4ea4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.617+0000 7f4ebb7fe700 1 -- 172.21.15.38:0/3845301962 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f4eac018070 con 0x7f4ec40c3180 2024-08-15T18:01:55.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.617+0000 7f4ebb7fe700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] conn(0x7f4ea4066750 0x7f4ea4068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.617+0000 7f4ebb7fe700 1 -- 172.21.15.38:0/3845301962 --> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ea4069210 con 0x7f4ea4066750 2024-08-15T18:01:55.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.617+0000 7f4ebb7fe700 1 -- 172.21.15.38:0/3845301962 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eac07dab0 con 0x7f4ec40c3180 2024-08-15T18:01:55.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.617+0000 7f4ec2ffd700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4ea4060860 0x7f4ea4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.617+0000 7f4ec37fe700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] conn(0x7f4ea4066750 0x7f4ea4068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.617+0000 7f4ec37fe700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] conn(0x7f4ea4066750 0x7f4ea4068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:55.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.617+0000 7f4ec2ffd700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4ea4060860 0x7f4ea4062d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4001ed0 tx=0x7f4eb4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:55.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.617+0000 7f4ebb7fe700 1 -- 172.21.15.38:0/3845301962 <== osd.0 v2:172.21.15.38:6808/24121 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f4ea4069210 con 0x7f4ea4066750 2024-08-15T18:01:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.629+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3845301962 --> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4ec406ece0 con 0x7f4ea4066750 2024-08-15T18:01:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ebb7fe700 1 -- 172.21.15.38:0/3845301962 <== osd.0 v2:172.21.15.38:6808/24121 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4ec406ece0 con 0x7f4ea4066750 2024-08-15T18:01:55.635 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T18:01:55.636 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T18:01:55.636 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T18:01:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3845301962 >> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] conn(0x7f4ea4066750 msgr2=0x7f4ea4068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ec3fff700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.38:6808/24121,v1:172.21.15.38:6809/24121] conn(0x7f4ea4066750 0x7f4ea4068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4ea4060860 msgr2=0x7f4ea4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ec3fff700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4ea4060860 0x7f4ea4062d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4001ed0 tx=0x7f4eb4009040 comp rx=0 tx=0).stop 2024-08-15T18:01:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3845301962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec40c3180 msgr2=0x7f4ec40c35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ec3fff700 1 --2- 172.21.15.38:0/3845301962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec40c3180 0x7f4ec40c35f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4eac011d70 tx=0x7f4eac0091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ec37fe700 1 -- 172.21.15.38:0/3845301962 reap_dead start 2024-08-15T18:01:55.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3845301962 shutdown_connections 2024-08-15T18:01:55.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3845301962 >> 172.21.15.38:0/3845301962 conn(0x7f4ec40ccfa0 msgr2=0x7f4ec40c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:55.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3845301962 shutdown_connections 2024-08-15T18:01:55.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.633+0000 7f4ec3fff700 1 -- 172.21.15.38:0/3845301962 wait complete. 2024-08-15T18:01:55.648 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-08-15T18:01:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ec5fb6700 1 Processor -- start 2024-08-15T18:01:55.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ec5fb6700 1 -- start start 2024-08-15T18:01:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ec5fb6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f4ec0159570 0x7f4ec00b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ec5fb6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4ec0159940 0x7f4ec00b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ec5fb6700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec00b0fc0 con 0x7f4ec00b3d20 2024-08-15T18:01:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ec5fb6700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec00b1860 con 0x7f4ec0159940 2024-08-15T18:01:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ec5fb6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec00b12a0 con 0x7f4ec0159570 2024-08-15T18:01:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebeffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4ec0159940 0x7f4ec00b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebeffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4ec0159940 0x7f4ec00b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48492/0 (socket says 172.21.15.38:48492) 2024-08-15T18:01:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebeffd700 1 -- 172.21.15.38:0/1076847419 learned_addr learned my addr 172.21.15.38:0/1076847419 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebf7fe700 1 --2- 172.21.15.38:0/1076847419 >> v2:172.21.15.104:3300/0 conn(0x7f4ec0159570 0x7f4ec00b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebeffd700 1 -- 172.21.15.38:0/1076847419 >> v2:172.21.15.104:3300/0 conn(0x7f4ec0159570 msgr2=0x7f4ec00b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebeffd700 1 --2- 172.21.15.38:0/1076847419 >> v2:172.21.15.104:3300/0 conn(0x7f4ec0159570 0x7f4ec00b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebeffd700 1 -- 172.21.15.38:0/1076847419 >> v1:172.21.15.104:6789/0 conn(0x7f4ec00b3d20 legacy=0x7f4ec00b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebeffd700 1 -- 172.21.15.38:0/1076847419 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00b1580 con 0x7f4ec0159940 2024-08-15T18:01:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebeffd700 1 --2- 172.21.15.38:0/1076847419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec0159940 0x7f4ec00b0770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0070830 tx=0x7f4eb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8819219a33a53f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebe7fc700 1 -- 172.21.15.38:0/1076847419 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000b860 con 0x7f4ec0159940 2024-08-15T18:01:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebe7fc700 1 -- 172.21.15.38:0/1076847419 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb000b9c0 con 0x7f4ec0159940 2024-08-15T18:01:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.765+0000 7f4ebe7fc700 1 -- 172.21.15.38:0/1076847419 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000bd60 con 0x7f4ec0159940 2024-08-15T18:01:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.769+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/1076847419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec0159940 msgr2=0x7f4ec00b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.769+0000 7f4ec5fb6700 1 --2- 172.21.15.38:0/1076847419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec0159940 0x7f4ec00b0770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0070830 tx=0x7f4eb0009250 comp rx=0 tx=0).stop 2024-08-15T18:01:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.769+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/1076847419 shutdown_connections 2024-08-15T18:01:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.769+0000 7f4ec5fb6700 1 --2- 172.21.15.38:0/1076847419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec0159940 0x7f4ec00b0770 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.769+0000 7f4ec5fb6700 1 --2- 172.21.15.38:0/1076847419 >> v2:172.21.15.104:3300/0 conn(0x7f4ec0159570 0x7f4ec00b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.769+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/1076847419 >> 172.21.15.38:0/1076847419 conn(0x7f4ec00996f0 msgr2=0x7f4ec0067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:55.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.769+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/1076847419 shutdown_connections 2024-08-15T18:01:55.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.769+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/1076847419 wait complete. 2024-08-15T18:01:55.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ec5fb6700 1 Processor -- start 2024-08-15T18:01:55.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ec5fb6700 1 -- start start 2024-08-15T18:01:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ec5fb6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec00b3d20 0x7f4ec01412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ec5fb6700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ec0159570 0x7f4ec009bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ec5fb6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec0159940 0x7f4ec009c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ec5fb6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec009d7c0 con 0x7f4ec00b3d20 2024-08-15T18:01:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ec5fb6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec009d330 con 0x7f4ec0159940 2024-08-15T18:01:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ec5fb6700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0150d20 con 0x7f4ec0159570 2024-08-15T18:01:55.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec00b3d20 0x7f4ec01412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec00b3d20 0x7f4ec01412b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48508/0 (socket says 172.21.15.38:48508) 2024-08-15T18:01:55.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebf7fe700 1 -- 172.21.15.38:0/2551075394 learned_addr learned my addr 172.21.15.38:0/2551075394 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:55.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebffff700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec0159940 0x7f4ec009c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebeffd700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ec0159570 0x7f4ec009bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebf7fe700 1 -- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ec0159570 msgr2=0x7f4ec009bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebf7fe700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ec0159570 0x7f4ec009bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebf7fe700 1 -- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec0159940 msgr2=0x7f4ec009c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebf7fe700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec0159940 0x7f4ec009c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebf7fe700 1 -- 172.21.15.38:0/2551075394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb00094f0 con 0x7f4ec00b3d20 2024-08-15T18:01:55.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebffff700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec0159940 0x7f4ec009c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:55.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ebf7fe700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec00b3d20 0x7f4ec01412b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800e190 tx=0x7f4ea8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:55.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4eb77fe700 1 -- 172.21.15.38:0/2551075394 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea800ed80 con 0x7f4ec00b3d20 2024-08-15T18:01:55.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4eb77fe700 1 -- 172.21.15.38:0/2551075394 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea8014420 con 0x7f4ec00b3d20 2024-08-15T18:01:55.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/2551075394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec009c720 con 0x7f4ec00b3d20 2024-08-15T18:01:55.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.773+0000 7f4eb77fe700 1 -- 172.21.15.38:0/2551075394 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80147e0 con 0x7f4ec00b3d20 2024-08-15T18:01:55.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.777+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/2551075394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec009cce0 con 0x7f4ec00b3d20 2024-08-15T18:01:55.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.777+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/2551075394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec013fad0 con 0x7f4ec00b3d20 2024-08-15T18:01:55.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.777+0000 7f4eb77fe700 1 -- 172.21.15.38:0/2551075394 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4ea8014b50 con 0x7f4ec00b3d20 2024-08-15T18:01:55.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.777+0000 7f4eb77fe700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4ea0060860 0x7f4ea0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:55.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.777+0000 7f4eb77fe700 1 -- 172.21.15.38:0/2551075394 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f4ea807d0a0 con 0x7f4ec00b3d20 2024-08-15T18:01:55.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.777+0000 7f4ebeffd700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4ea0060860 0x7f4ea0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:55.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.781+0000 7f4ebeffd700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4ea0060860 0x7f4ea0062d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0070830 tx=0x7f4eb0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:55.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.789+0000 7f4eb77fe700 1 -- 172.21.15.38:0/2551075394 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4ea804d790 con 0x7f4ec00b3d20 2024-08-15T18:01:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.953+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/2551075394 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ec009d470 con 0x7f4ea0060860 2024-08-15T18:01:55.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.953+0000 7f4eb77fe700 1 -- 172.21.15.38:0/2551075394 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4eb000bb30 con 0x7f4ea0060860 2024-08-15T18:01:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4ea0060860 msgr2=0x7f4ea0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4ea0060860 0x7f4ea0062d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0070830 tx=0x7f4eb0017000 comp rx=0 tx=0).stop 2024-08-15T18:01:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/2551075394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec00b3d20 msgr2=0x7f4ec01412b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec00b3d20 0x7f4ec01412b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800e190 tx=0x7f4ea8009920 comp rx=0 tx=0).stop 2024-08-15T18:01:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/2551075394 shutdown_connections 2024-08-15T18:01:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f4ea0060860 0x7f4ea0062d10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec0159940 0x7f4ec009c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ec0159570 0x7f4ec009bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 --2- 172.21.15.38:0/2551075394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ec00b3d20 0x7f4ec01412b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/2551075394 >> 172.21.15.38:0/2551075394 conn(0x7f4ec00996f0 msgr2=0x7f4ec00b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/2551075394 shutdown_connections 2024-08-15T18:01:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:55.957+0000 7f4ec5fb6700 1 -- 172.21.15.38:0/2551075394 wait complete. 2024-08-15T18:01:55.965 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 0 to deep-scrub 2024-08-15T18:01:55.979 INFO:tasks.ceph:Scrubbing osd.1 2024-08-15T18:01:55.979 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-08-15T18:01:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.089+0000 7f7b48a4f700 1 Processor -- start 2024-08-15T18:01:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.089+0000 7f7b48a4f700 1 -- start start 2024-08-15T18:01:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7b440b2e70 0x7f7b440cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7b440cc230 0x7f7b440c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b440c5bb0 con 0x7f7b440ce640 2024-08-15T18:01:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b440b32f0 con 0x7f7b440cc230 2024-08-15T18:01:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b440c4fa0 con 0x7f7b440b2e70 2024-08-15T18:01:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b43fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7b440cc230 0x7f7b440c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b437fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7b440ce640 0x7f7b440cea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40430/0 (socket says 172.21.15.38:40430) 2024-08-15T18:01:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b437fe700 1 -- 172.21.15.38:0/4286150674 learned_addr learned my addr 172.21.15.38:0/4286150674 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b42ffd700 1 --2- 172.21.15.38:0/4286150674 >> v2:172.21.15.104:3300/0 conn(0x7f7b440b2e70 0x7f7b440cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b43fff700 1 -- 172.21.15.38:0/4286150674 >> v2:172.21.15.104:3300/0 conn(0x7f7b440b2e70 msgr2=0x7f7b440cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b43fff700 1 --2- 172.21.15.38:0/4286150674 >> v2:172.21.15.104:3300/0 conn(0x7f7b440b2e70 0x7f7b440cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b43fff700 1 -- 172.21.15.38:0/4286150674 >> v1:172.21.15.38:6789/0 conn(0x7f7b440ce640 legacy=0x7f7b440cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b43fff700 1 -- 172.21.15.38:0/4286150674 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b440c5280 con 0x7f7b440cc230 2024-08-15T18:01:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b42ffd700 1 --2- 172.21.15.38:0/4286150674 >> v2:172.21.15.104:3300/0 conn(0x7f7b440b2e70 0x7f7b440cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:01:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b43fff700 1 --2- 172.21.15.38:0/4286150674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b440cc230 0x7f7b440c6040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7b440ca6a0 tx=0x7f7b3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed177a038c73b5a server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b427fc700 1 -- 172.21.15.38:0/4286150674 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c00e070 con 0x7f7b440cc230 2024-08-15T18:01:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b427fc700 1 -- 172.21.15.38:0/4286150674 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c00bb60 con 0x7f7b440cc230 2024-08-15T18:01:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b427fc700 1 -- 172.21.15.38:0/4286150674 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c012430 con 0x7f7b440cc230 2024-08-15T18:01:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 -- 172.21.15.38:0/4286150674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b440cc230 msgr2=0x7f7b440c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 --2- 172.21.15.38:0/4286150674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b440cc230 0x7f7b440c6040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7b440ca6a0 tx=0x7f7b3c009780 comp rx=0 tx=0).stop 2024-08-15T18:01:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 -- 172.21.15.38:0/4286150674 shutdown_connections 2024-08-15T18:01:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 --2- 172.21.15.38:0/4286150674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b440cc230 0x7f7b440c6040 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 --2- 172.21.15.38:0/4286150674 >> v2:172.21.15.104:3300/0 conn(0x7f7b440b2e70 0x7f7b440cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 -- 172.21.15.38:0/4286150674 >> 172.21.15.38:0/4286150674 conn(0x7f7b44099410 msgr2=0x7f7b440675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:56.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 -- 172.21.15.38:0/4286150674 shutdown_connections 2024-08-15T18:01:56.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.093+0000 7f7b48a4f700 1 -- 172.21.15.38:0/4286150674 wait complete. 2024-08-15T18:01:56.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b48a4f700 1 Processor -- start 2024-08-15T18:01:56.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b48a4f700 1 -- start start 2024-08-15T18:01:56.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b48a4f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b440b2e70 0x7f7b44140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b48a4f700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b440cc230 0x7f7b440c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b48a4f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b440ce640 0x7f7b44154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b48a4f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b44157e80 con 0x7f7b440ce640 2024-08-15T18:01:56.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b48a4f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b44158160 con 0x7f7b440b2e70 2024-08-15T18:01:56.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b48a4f700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b44158440 con 0x7f7b440cc230 2024-08-15T18:01:56.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b43fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b440ce640 0x7f7b44154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b437fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b440b2e70 0x7f7b44140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b43fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b440ce640 0x7f7b44154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48524/0 (socket says 172.21.15.38:48524) 2024-08-15T18:01:56.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b43fff700 1 -- 172.21.15.38:0/1963967588 learned_addr learned my addr 172.21.15.38:0/1963967588 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:56.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b42ffd700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b440cc230 0x7f7b440c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b43fff700 1 -- 172.21.15.38:0/1963967588 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b440cc230 msgr2=0x7f7b440c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b43fff700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b440cc230 0x7f7b440c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b43fff700 1 -- 172.21.15.38:0/1963967588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b440b2e70 msgr2=0x7f7b44140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b43fff700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b440b2e70 0x7f7b44140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b43fff700 1 -- 172.21.15.38:0/1963967588 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b34009420 con 0x7f7b440ce640 2024-08-15T18:01:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b437fe700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b440b2e70 0x7f7b44140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:01:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.097+0000 7f7b43fff700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b440ce640 0x7f7b44154f10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7b44067de0 tx=0x7f7b3c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b2b7fe700 1 -- 172.21.15.38:0/1963967588 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0070a0 con 0x7f7b440ce640 2024-08-15T18:01:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b2b7fe700 1 -- 172.21.15.38:0/1963967588 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c0263f0 con 0x7f7b440ce640 2024-08-15T18:01:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b48a4f700 1 -- 172.21.15.38:0/1963967588 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3c009a20 con 0x7f7b440ce640 2024-08-15T18:01:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b2b7fe700 1 -- 172.21.15.38:0/1963967588 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0267b0 con 0x7f7b440ce640 2024-08-15T18:01:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b48a4f700 1 -- 172.21.15.38:0/1963967588 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b44157a70 con 0x7f7b440ce640 2024-08-15T18:01:56.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b48a4f700 1 -- 172.21.15.38:0/1963967588 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b440644b0 con 0x7f7b440ce640 2024-08-15T18:01:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b2b7fe700 1 -- 172.21.15.38:0/1963967588 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7b3c026be0 con 0x7f7b440ce640 2024-08-15T18:01:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b2b7fe700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7b30060890 0x7f7b30062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b2b7fe700 1 -- 172.21.15.38:0/1963967588 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f7b3c07ee50 con 0x7f7b440ce640 2024-08-15T18:01:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b2b7fe700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] conn(0x7f7b30066780 0x7f7b30068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b2b7fe700 1 -- 172.21.15.38:0/1963967588 --> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b30069240 con 0x7f7b30066780 2024-08-15T18:01:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.101+0000 7f7b2b7fe700 1 -- 172.21.15.38:0/1963967588 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b3c07f220 con 0x7f7b440ce640 2024-08-15T18:01:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.105+0000 7f7b437fe700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7b30060890 0x7f7b30062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.105+0000 7f7b42ffd700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] conn(0x7f7b30066780 0x7f7b30068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.105+0000 7f7b42ffd700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] conn(0x7f7b30066780 0x7f7b30068b90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.105+0000 7f7b437fe700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7b30060890 0x7f7b30062d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b38000c00 tx=0x7f7b38009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.105+0000 7f7b2b7fe700 1 -- 172.21.15.38:0/1963967588 <== osd.1 v2:172.21.15.38:6816/24122 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f7b30069240 con 0x7f7b30066780 2024-08-15T18:01:56.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b48a4f700 1 -- 172.21.15.38:0/1963967588 --> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7b4406f200 con 0x7f7b30066780 2024-08-15T18:01:56.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b2b7fe700 1 -- 172.21.15.38:0/1963967588 <== osd.1 v2:172.21.15.38:6816/24122 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7b4406f200 con 0x7f7b30066780 2024-08-15T18:01:56.125 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T18:01:56.125 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T18:01:56.125 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T18:01:56.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b48a4f700 1 -- 172.21.15.38:0/1963967588 >> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] conn(0x7f7b30066780 msgr2=0x7f7b30068b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b48a4f700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.38:6816/24122,v1:172.21.15.38:6817/24122] conn(0x7f7b30066780 0x7f7b30068b90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b48a4f700 1 -- 172.21.15.38:0/1963967588 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7b30060890 msgr2=0x7f7b30062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b48a4f700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7b30060890 0x7f7b30062d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b38000c00 tx=0x7f7b38009040 comp rx=0 tx=0).stop 2024-08-15T18:01:56.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b48a4f700 1 -- 172.21.15.38:0/1963967588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b440ce640 msgr2=0x7f7b44154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b48a4f700 1 --2- 172.21.15.38:0/1963967588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b440ce640 0x7f7b44154f10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7b44067de0 tx=0x7f7b3c019000 comp rx=0 tx=0).stop 2024-08-15T18:01:56.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b43fff700 1 -- 172.21.15.38:0/1963967588 reap_dead start 2024-08-15T18:01:56.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b48a4f700 1 -- 172.21.15.38:0/1963967588 shutdown_connections 2024-08-15T18:01:56.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b48a4f700 1 -- 172.21.15.38:0/1963967588 >> 172.21.15.38:0/1963967588 conn(0x7f7b44099410 msgr2=0x7f7b440b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:56.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b48a4f700 1 -- 172.21.15.38:0/1963967588 shutdown_connections 2024-08-15T18:01:56.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.121+0000 7f7b48a4f700 1 -- 172.21.15.38:0/1963967588 wait complete. 2024-08-15T18:01:56.142 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-08-15T18:01:56.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.245+0000 7fac86f15700 1 Processor -- start 2024-08-15T18:01:56.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.245+0000 7fac86f15700 1 -- start start 2024-08-15T18:01:56.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.245+0000 7fac86f15700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fac800cd1b0 0x7fac801536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.245+0000 7fac86f15700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fac8015a190 0x7fac8015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.245+0000 7fac86f15700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8015a610 con 0x7fac80153320 2024-08-15T18:01:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.245+0000 7fac86f15700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac800ce430 con 0x7fac8015a190 2024-08-15T18:01:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.245+0000 7fac86f15700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac8015dbd0 con 0x7fac800cd1b0 2024-08-15T18:01:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.245+0000 7fac85f13700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fac80153320 0x7fac800ccaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40432/0 (socket says 172.21.15.38:40432) 2024-08-15T18:01:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.245+0000 7fac85f13700 1 -- 172.21.15.38:0/1299005201 learned_addr learned my addr 172.21.15.38:0/1299005201 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86714700 1 --2- 172.21.15.38:0/1299005201 >> v2:172.21.15.38:3300/0 conn(0x7fac8015a190 0x7fac8015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac85712700 1 --2- 172.21.15.38:0/1299005201 >> v2:172.21.15.104:3300/0 conn(0x7fac800cd1b0 0x7fac801536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac84f11700 1 -- 172.21.15.38:0/1299005201 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805440570 0 0) 0x7fac8015a610 con 0x7fac80153320 2024-08-15T18:01:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac84f11700 1 -- 172.21.15.38:0/1299005201 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac6c003570 con 0x7fac80153320 2024-08-15T18:01:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86714700 1 -- 172.21.15.38:0/1299005201 >> v2:172.21.15.104:3300/0 conn(0x7fac800cd1b0 msgr2=0x7fac801536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86714700 1 --2- 172.21.15.38:0/1299005201 >> v2:172.21.15.104:3300/0 conn(0x7fac800cd1b0 0x7fac801536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86714700 1 -- 172.21.15.38:0/1299005201 >> v1:172.21.15.38:6789/0 conn(0x7fac80153320 legacy=0x7fac800ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86714700 1 -- 172.21.15.38:0/1299005201 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8015d370 con 0x7fac8015a190 2024-08-15T18:01:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86714700 1 --2- 172.21.15.38:0/1299005201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac8015a190 0x7fac8015cc60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fac78009e60 tx=0x7fac78009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652de9d23dedbcbb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac84f11700 1 -- 172.21.15.38:0/1299005201 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7800e070 con 0x7fac8015a190 2024-08-15T18:01:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac84f11700 1 -- 172.21.15.38:0/1299005201 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7800ba70 con 0x7fac8015a190 2024-08-15T18:01:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac84f11700 1 -- 172.21.15.38:0/1299005201 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7800be30 con 0x7fac8015a190 2024-08-15T18:01:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86f15700 1 -- 172.21.15.38:0/1299005201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac8015a190 msgr2=0x7fac8015cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86f15700 1 --2- 172.21.15.38:0/1299005201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac8015a190 0x7fac8015cc60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fac78009e60 tx=0x7fac78009780 comp rx=0 tx=0).stop 2024-08-15T18:01:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86f15700 1 -- 172.21.15.38:0/1299005201 shutdown_connections 2024-08-15T18:01:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86f15700 1 --2- 172.21.15.38:0/1299005201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac8015a190 0x7fac8015cc60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86f15700 1 --2- 172.21.15.38:0/1299005201 >> v2:172.21.15.104:3300/0 conn(0x7fac800cd1b0 0x7fac801536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86f15700 1 -- 172.21.15.38:0/1299005201 >> 172.21.15.38:0/1299005201 conn(0x7fac800d00a0 msgr2=0x7fac800d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86f15700 1 -- 172.21.15.38:0/1299005201 shutdown_connections 2024-08-15T18:01:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86f15700 1 -- 172.21.15.38:0/1299005201 wait complete. 2024-08-15T18:01:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86f15700 1 Processor -- start 2024-08-15T18:01:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.249+0000 7fac86f15700 1 -- start start 2024-08-15T18:01:56.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac86f15700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800cd1b0 0x7fac800b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac86f15700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac80153320 0x7fac800b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac86f15700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac8015a190 0x7fac80142b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac86f15700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac800b5870 con 0x7fac800cd1b0 2024-08-15T18:01:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac86f15700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac800c7280 con 0x7fac8015a190 2024-08-15T18:01:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac86f15700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac80143080 con 0x7fac80153320 2024-08-15T18:01:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac85f13700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800cd1b0 0x7fac800b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac85f13700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800cd1b0 0x7fac800b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48534/0 (socket says 172.21.15.38:48534) 2024-08-15T18:01:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac85f13700 1 -- 172.21.15.38:0/1934876256 learned_addr learned my addr 172.21.15.38:0/1934876256 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac85712700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac80153320 0x7fac800b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac86714700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac8015a190 0x7fac80142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac85f13700 1 -- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac80153320 msgr2=0x7fac800b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac85f13700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac80153320 0x7fac800b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac85f13700 1 -- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac8015a190 msgr2=0x7fac80142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac85f13700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac8015a190 0x7fac80142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac85f13700 1 -- 172.21.15.38:0/1934876256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac78009a20 con 0x7fac800cd1b0 2024-08-15T18:01:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac85f13700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800cd1b0 0x7fac800b2e40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00c7c0 tx=0x7fac7c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac71ffb700 1 -- 172.21.15.38:0/1934876256 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c005cd0 con 0x7fac800cd1b0 2024-08-15T18:01:56.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac71ffb700 1 -- 172.21.15.38:0/1934876256 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c005e30 con 0x7fac800cd1b0 2024-08-15T18:01:56.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac71ffb700 1 -- 172.21.15.38:0/1934876256 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c007320 con 0x7fac800cd1b0 2024-08-15T18:01:56.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac86f15700 1 -- 172.21.15.38:0/1934876256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac800b3dc0 con 0x7fac800cd1b0 2024-08-15T18:01:56.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.253+0000 7fac86f15700 1 -- 172.21.15.38:0/1934876256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac800b3800 con 0x7fac800cd1b0 2024-08-15T18:01:56.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.257+0000 7fac86f15700 1 -- 172.21.15.38:0/1934876256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac800638f0 con 0x7fac800cd1b0 2024-08-15T18:01:56.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.257+0000 7fac71ffb700 1 -- 172.21.15.38:0/1934876256 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fac7c01b3b0 con 0x7fac800cd1b0 2024-08-15T18:01:56.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.257+0000 7fac71ffb700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fac64060860 0x7fac64062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.257+0000 7fac71ffb700 1 -- 172.21.15.38:0/1934876256 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fac7c07c060 con 0x7fac800cd1b0 2024-08-15T18:01:56.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.257+0000 7fac85712700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fac64060860 0x7fac64062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.257+0000 7fac85712700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fac64060860 0x7fac64062d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac800cc9f0 tx=0x7fac740091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.261+0000 7fac71ffb700 1 -- 172.21.15.38:0/1934876256 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fac7c04c6b0 con 0x7fac800cd1b0 2024-08-15T18:01:56.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 -- 172.21.15.38:0/1934876256 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fac800c73c0 con 0x7fac64060860 2024-08-15T18:01:56.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac71ffb700 1 -- 172.21.15.38:0/1934876256 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fac74011a20 con 0x7fac64060860 2024-08-15T18:01:56.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 -- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fac64060860 msgr2=0x7fac64062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fac64060860 0x7fac64062d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac800cc9f0 tx=0x7fac740091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 -- 172.21.15.38:0/1934876256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800cd1b0 msgr2=0x7fac800b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800cd1b0 0x7fac800b2e40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00c7c0 tx=0x7fac7c007e00 comp rx=0 tx=0).stop 2024-08-15T18:01:56.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 -- 172.21.15.38:0/1934876256 shutdown_connections 2024-08-15T18:01:56.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fac64060860 0x7fac64062d10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac8015a190 0x7fac80142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac80153320 0x7fac800b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 --2- 172.21.15.38:0/1934876256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac800cd1b0 0x7fac800b2e40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 -- 172.21.15.38:0/1934876256 >> 172.21.15.38:0/1934876256 conn(0x7fac800d00a0 msgr2=0x7fac800c63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:56.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 -- 172.21.15.38:0/1934876256 shutdown_connections 2024-08-15T18:01:56.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.429+0000 7fac86f15700 1 -- 172.21.15.38:0/1934876256 wait complete. 2024-08-15T18:01:56.437 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 1 to deep-scrub 2024-08-15T18:01:56.447 INFO:tasks.ceph:Scrubbing osd.2 2024-08-15T18:01:56.447 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-08-15T18:01:56.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.561+0000 7f6791310700 1 Processor -- start 2024-08-15T18:01:56.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.561+0000 7f6791310700 1 -- start start 2024-08-15T18:01:56.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f678c158fe0 0x7f678c0abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f678c1593b0 0x7f678c159800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c159f20 con 0x7f678c0686a0 2024-08-15T18:01:56.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c159d40 con 0x7f678c1593b0 2024-08-15T18:01:56.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c15a200 con 0x7f678c158fe0 2024-08-15T18:01:56.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f678b7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f678c1593b0 0x7f678c159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6790b0f700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f678c0686a0 0x7f678c15cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.38:46690/0 (socket says 172.21.15.38:46690) 2024-08-15T18:01:56.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6790b0f700 1 -- 172.21.15.38:0/74039087 learned_addr learned my addr 172.21.15.38:0/74039087 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:56.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f678b7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f678c1593b0 0x7f678c159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48536/0 (socket says 172.21.15.38:48536) 2024-08-15T18:01:56.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f678bfff700 1 --2- 172.21.15.38:0/74039087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f678c158fe0 0x7f678c0abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f678b7fe700 1 -- 172.21.15.38:0/74039087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f678c158fe0 msgr2=0x7f678c0abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:56.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f678b7fe700 1 --2- 172.21.15.38:0/74039087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f678c158fe0 0x7f678c0abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f678b7fe700 1 -- 172.21.15.38:0/74039087 >> v1:172.21.15.104:6789/0 conn(0x7f678c0686a0 legacy=0x7f678c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f678b7fe700 1 -- 172.21.15.38:0/74039087 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c15a4e0 con 0x7f678c1593b0 2024-08-15T18:01:56.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f678b7fe700 1 --2- 172.21.15.38:0/74039087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f678c1593b0 0x7f678c159800 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f67740098e0 tx=0x7f67740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f11d18af98fb05 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f678affd700 1 -- 172.21.15.38:0/74039087 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677400b640 con 0x7f678c1593b0 2024-08-15T18:01:56.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f678affd700 1 -- 172.21.15.38:0/74039087 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677400b7a0 con 0x7f678c1593b0 2024-08-15T18:01:56.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f678affd700 1 -- 172.21.15.38:0/74039087 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677400bb40 con 0x7f678c1593b0 2024-08-15T18:01:56.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 -- 172.21.15.38:0/74039087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f678c1593b0 msgr2=0x7f678c159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 --2- 172.21.15.38:0/74039087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f678c1593b0 0x7f678c159800 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f67740098e0 tx=0x7f67740091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 -- 172.21.15.38:0/74039087 shutdown_connections 2024-08-15T18:01:56.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 --2- 172.21.15.38:0/74039087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f678c1593b0 0x7f678c159800 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 --2- 172.21.15.38:0/74039087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f678c158fe0 0x7f678c0abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 -- 172.21.15.38:0/74039087 >> 172.21.15.38:0/74039087 conn(0x7f678c0ccfe0 msgr2=0x7f678c06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:56.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 -- 172.21.15.38:0/74039087 shutdown_connections 2024-08-15T18:01:56.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.565+0000 7f6791310700 1 -- 172.21.15.38:0/74039087 wait complete. 2024-08-15T18:01:56.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6791310700 1 Processor -- start 2024-08-15T18:01:56.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6791310700 1 -- start start 2024-08-15T18:01:56.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6791310700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f678c0686a0 0x7f678c16a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6791310700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f678c158fe0 0x7f678c0c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6791310700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f678c1593b0 0x7f678c0c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6791310700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0cc300 con 0x7f678c1593b0 2024-08-15T18:01:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6791310700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c16a7b0 con 0x7f678c158fe0 2024-08-15T18:01:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6791310700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0c57b0 con 0x7f678c0686a0 2024-08-15T18:01:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6790b0f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f678c1593b0 0x7f678c0c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6790b0f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f678c1593b0 0x7f678c0c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48542/0 (socket says 172.21.15.38:48542) 2024-08-15T18:01:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6790b0f700 1 -- 172.21.15.38:0/3503851234 learned_addr learned my addr 172.21.15.38:0/3503851234 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f678b7fe700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f678c158fe0 0x7f678c0c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f678bfff700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f678c0686a0 0x7f678c16a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6790b0f700 1 -- 172.21.15.38:0/3503851234 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f678c0686a0 msgr2=0x7f678c16a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6790b0f700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f678c0686a0 0x7f678c16a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6790b0f700 1 -- 172.21.15.38:0/3503851234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f678c158fe0 msgr2=0x7f678c0c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6790b0f700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f678c158fe0 0x7f678c0c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6790b0f700 1 -- 172.21.15.38:0/3503851234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6774009420 con 0x7f678c1593b0 2024-08-15T18:01:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6790b0f700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f678c1593b0 0x7f678c0c75f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f678400cc40 tx=0x7f6784001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6783fff700 1 -- 172.21.15.38:0/3503851234 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400a910 con 0x7f678c1593b0 2024-08-15T18:01:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6783fff700 1 -- 172.21.15.38:0/3503851234 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678400aa70 con 0x7f678c1593b0 2024-08-15T18:01:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6791310700 1 -- 172.21.15.38:0/3503851234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c0c5a90 con 0x7f678c1593b0 2024-08-15T18:01:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6783fff700 1 -- 172.21.15.38:0/3503851234 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400ae30 con 0x7f678c1593b0 2024-08-15T18:01:56.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.569+0000 7f6791310700 1 -- 172.21.15.38:0/3503851234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c0c6050 con 0x7f678c1593b0 2024-08-15T18:01:56.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.573+0000 7f6791310700 1 -- 172.21.15.38:0/3503851234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f678c0644b0 con 0x7f678c1593b0 2024-08-15T18:01:56.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.573+0000 7f6783fff700 1 -- 172.21.15.38:0/3503851234 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6784007300 con 0x7f678c1593b0 2024-08-15T18:01:56.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.573+0000 7f6783fff700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f676c060860 0x7f676c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.573+0000 7f6783fff700 1 -- 172.21.15.38:0/3503851234 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f678407cd10 con 0x7f678c1593b0 2024-08-15T18:01:56.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.573+0000 7f678bfff700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f676c060860 0x7f676c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.573+0000 7f6783fff700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] conn(0x7f676c066750 0x7f676c068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.573+0000 7f6783fff700 1 -- 172.21.15.38:0/3503851234 --> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f676c069210 con 0x7f676c066750 2024-08-15T18:01:56.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.573+0000 7f6783fff700 1 -- 172.21.15.38:0/3503851234 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f678407f270 con 0x7f678c1593b0 2024-08-15T18:01:56.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.573+0000 7f678b7fe700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] conn(0x7f676c066750 0x7f676c068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.573+0000 7f678bfff700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f676c060860 0x7f676c062d10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f678c0b10c0 tx=0x7f677c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.573+0000 7f678b7fe700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] conn(0x7f676c066750 0x7f676c068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.577+0000 7f6783fff700 1 -- 172.21.15.38:0/3503851234 <== osd.2 v2:172.21.15.38:6824/24123 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f676c069210 con 0x7f676c066750 2024-08-15T18:01:56.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6791310700 1 -- 172.21.15.38:0/3503851234 --> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f678c15a840 con 0x7f676c066750 2024-08-15T18:01:56.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6783fff700 1 -- 172.21.15.38:0/3503851234 <== osd.2 v2:172.21.15.38:6824/24123 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f678c15a840 con 0x7f676c066750 2024-08-15T18:01:56.598 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T18:01:56.598 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T18:01:56.598 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T18:01:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6791310700 1 -- 172.21.15.38:0/3503851234 >> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] conn(0x7f676c066750 msgr2=0x7f676c068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6791310700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.38:6824/24123,v1:172.21.15.38:6825/24123] conn(0x7f676c066750 0x7f676c068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6791310700 1 -- 172.21.15.38:0/3503851234 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f676c060860 msgr2=0x7f676c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6791310700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f676c060860 0x7f676c062d10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f678c0b10c0 tx=0x7f677c008040 comp rx=0 tx=0).stop 2024-08-15T18:01:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6791310700 1 -- 172.21.15.38:0/3503851234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f678c1593b0 msgr2=0x7f678c0c75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6791310700 1 --2- 172.21.15.38:0/3503851234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f678c1593b0 0x7f678c0c75f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f678400cc40 tx=0x7f6784001a50 comp rx=0 tx=0).stop 2024-08-15T18:01:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6790b0f700 1 -- 172.21.15.38:0/3503851234 reap_dead start 2024-08-15T18:01:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6791310700 1 -- 172.21.15.38:0/3503851234 shutdown_connections 2024-08-15T18:01:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6791310700 1 -- 172.21.15.38:0/3503851234 >> 172.21.15.38:0/3503851234 conn(0x7f678c0ccfe0 msgr2=0x7f678c0aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6791310700 1 -- 172.21.15.38:0/3503851234 shutdown_connections 2024-08-15T18:01:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.593+0000 7f6791310700 1 -- 172.21.15.38:0/3503851234 wait complete. 2024-08-15T18:01:56.610 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-08-15T18:01:56.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07fabe9700 1 Processor -- start 2024-08-15T18:01:56.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07fabe9700 1 -- start start 2024-08-15T18:01:56.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07fabe9700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f07f4159040 0x7f07f40cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07fabe9700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f07f40d0010 0x7f07f40d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07fabe9700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f40cf190 con 0x7f07f415cea0 2024-08-15T18:01:56.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07fabe9700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f415d9f0 con 0x7f07f40d0010 2024-08-15T18:01:56.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07fabe9700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f40d09a0 con 0x7f07f4159040 2024-08-15T18:01:56.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07f93e6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f07f40d0010 0x7f07f40d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07f93e6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f07f40d0010 0x7f07f40d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37944/0 (socket says 172.21.15.38:37944) 2024-08-15T18:01:56.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07f93e6700 1 -- 172.21.15.38:0/2316998750 learned_addr learned my addr 172.21.15.38:0/2316998750 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:56.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07fa3e8700 1 --1- 172.21.15.38:0/2316998750 >> v1:172.21.15.38:6789/0 conn(0x7f07f415cea0 0x7f07f415d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41312/0 (socket says 172.21.15.38:41312) 2024-08-15T18:01:56.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07f9be7700 1 --2- 172.21.15.38:0/2316998750 >> v2:172.21.15.104:3300/0 conn(0x7f07f4159040 0x7f07f40cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07f93e6700 1 -- 172.21.15.38:0/2316998750 >> v2:172.21.15.104:3300/0 conn(0x7f07f4159040 msgr2=0x7f07f40cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:56.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.721+0000 7f07f93e6700 1 --2- 172.21.15.38:0/2316998750 >> v2:172.21.15.104:3300/0 conn(0x7f07f4159040 0x7f07f40cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07f93e6700 1 -- 172.21.15.38:0/2316998750 >> v1:172.21.15.38:6789/0 conn(0x7f07f415cea0 legacy=0x7f07f415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07f93e6700 1 -- 172.21.15.38:0/2316998750 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f40ce5e0 con 0x7f07f40d0010 2024-08-15T18:01:56.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07f93e6700 1 --2- 172.21.15.38:0/2316998750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f40d0010 0x7f07f40d0460 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f07f4070d70 tx=0x7f07e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153ad4b4bdda72c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07f8be5700 1 -- 172.21.15.38:0/2316998750 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8009e70 con 0x7f07f40d0010 2024-08-15T18:01:56.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07f8be5700 1 -- 172.21.15.38:0/2316998750 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e800b860 con 0x7f07f40d0010 2024-08-15T18:01:56.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07f8be5700 1 -- 172.21.15.38:0/2316998750 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e800bc00 con 0x7f07f40d0010 2024-08-15T18:01:56.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07fabe9700 1 -- 172.21.15.38:0/2316998750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f40d0010 msgr2=0x7f07f40d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07fabe9700 1 --2- 172.21.15.38:0/2316998750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f40d0010 0x7f07f40d0460 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f07f4070d70 tx=0x7f07e8009250 comp rx=0 tx=0).stop 2024-08-15T18:01:56.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07fabe9700 1 -- 172.21.15.38:0/2316998750 shutdown_connections 2024-08-15T18:01:56.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07fabe9700 1 --2- 172.21.15.38:0/2316998750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f40d0010 0x7f07f40d0460 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07fabe9700 1 --2- 172.21.15.38:0/2316998750 >> v2:172.21.15.104:3300/0 conn(0x7f07f4159040 0x7f07f40cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07fabe9700 1 -- 172.21.15.38:0/2316998750 >> 172.21.15.38:0/2316998750 conn(0x7f07f4068250 msgr2=0x7f07f4068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:56.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07fabe9700 1 -- 172.21.15.38:0/2316998750 shutdown_connections 2024-08-15T18:01:56.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.725+0000 7f07fabe9700 1 -- 172.21.15.38:0/2316998750 wait complete. 2024-08-15T18:01:56.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fabe9700 1 Processor -- start 2024-08-15T18:01:56.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fabe9700 1 -- start start 2024-08-15T18:01:56.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fabe9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07f40d0010 0x7f07f415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fabe9700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f07f4159040 0x7f07f415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fabe9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f415cea0 0x7f07f415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fabe9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f40b03d0 con 0x7f07f415cea0 2024-08-15T18:01:56.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fabe9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f40b1370 con 0x7f07f40d0010 2024-08-15T18:01:56.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fabe9700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f40b0ee0 con 0x7f07f4159040 2024-08-15T18:01:56.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fa3e8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f415cea0 0x7f07f415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fa3e8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f415cea0 0x7f07f415ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37954/0 (socket says 172.21.15.38:37954) 2024-08-15T18:01:56.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fa3e8700 1 -- 172.21.15.38:0/1471368636 learned_addr learned my addr 172.21.15.38:0/1471368636 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:56.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07f93e6700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f07f4159040 0x7f07f415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07f9be7700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07f40d0010 0x7f07f415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07f93e6700 1 -- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07f40d0010 msgr2=0x7f07f415a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07f93e6700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07f40d0010 0x7f07f415a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07f93e6700 1 -- 172.21.15.38:0/1471368636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f415cea0 msgr2=0x7f07f415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07f93e6700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f415cea0 0x7f07f415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07f93e6700 1 -- 172.21.15.38:0/1471368636 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80094f0 con 0x7f07f4159040 2024-08-15T18:01:56.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07f93e6700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f07f4159040 0x7f07f415a910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07f40b6050 tx=0x7f07e8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07e5ffb700 1 -- 172.21.15.38:0/1471368636 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e800bbe0 con 0x7f07f4159040 2024-08-15T18:01:56.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07e5ffb700 1 -- 172.21.15.38:0/1471368636 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e8000c90 con 0x7f07f4159040 2024-08-15T18:01:56.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fabe9700 1 -- 172.21.15.38:0/1471368636 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f41391a0 con 0x7f07f4159040 2024-08-15T18:01:56.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07e5ffb700 1 -- 172.21.15.38:0/1471368636 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8011650 con 0x7f07f4159040 2024-08-15T18:01:56.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.729+0000 7f07fabe9700 1 -- 172.21.15.38:0/1471368636 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f40b0890 con 0x7f07f4159040 2024-08-15T18:01:56.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.733+0000 7f07fabe9700 1 -- 172.21.15.38:0/1471368636 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f40643a0 con 0x7f07f4159040 2024-08-15T18:01:56.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.733+0000 7f07e5ffb700 1 -- 172.21.15.38:0/1471368636 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f07e8046b20 con 0x7f07f4159040 2024-08-15T18:01:56.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.733+0000 7f07e5ffb700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f07d80608e0 0x7f07d8062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:56.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.733+0000 7f07e5ffb700 1 -- 172.21.15.38:0/1471368636 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f07e807e1b0 con 0x7f07f4159040 2024-08-15T18:01:56.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.733+0000 7f07f9be7700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f07d80608e0 0x7f07d8062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:56.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.733+0000 7f07f9be7700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f07d80608e0 0x7f07d8062d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009d10 tx=0x7f07f00091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:56.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.745+0000 7f07e5ffb700 1 -- 172.21.15.38:0/1471368636 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f07e804e800 con 0x7f07f4159040 2024-08-15T18:01:56.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.913+0000 7f07fabe9700 1 -- 172.21.15.38:0/1471368636 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f07f40b1020 con 0x7f07d80608e0 2024-08-15T18:01:56.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.913+0000 7f07e5ffb700 1 -- 172.21.15.38:0/1471368636 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f07f40b1020 con 0x7f07d80608e0 2024-08-15T18:01:56.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 -- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f07d80608e0 msgr2=0x7f07d8062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f07d80608e0 0x7f07d8062d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009d10 tx=0x7f07f00091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 -- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f07f4159040 msgr2=0x7f07f415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:56.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f07f4159040 0x7f07f415a910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07f40b6050 tx=0x7f07e8017000 comp rx=0 tx=0).stop 2024-08-15T18:01:56.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 -- 172.21.15.38:0/1471368636 shutdown_connections 2024-08-15T18:01:56.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f07d80608e0 0x7f07d8062d90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f415cea0 0x7f07f415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f07f4159040 0x7f07f415a910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 --2- 172.21.15.38:0/1471368636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07f40d0010 0x7f07f415a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:56.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 -- 172.21.15.38:0/1471368636 >> 172.21.15.38:0/1471368636 conn(0x7f07f4068250 msgr2=0x7f07f4063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:56.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 -- 172.21.15.38:0/1471368636 shutdown_connections 2024-08-15T18:01:56.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:56.917+0000 7f07fabe9700 1 -- 172.21.15.38:0/1471368636 wait complete. 2024-08-15T18:01:56.923 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 2 to deep-scrub 2024-08-15T18:01:56.933 INFO:tasks.ceph:Scrubbing osd.3 2024-08-15T18:01:56.933 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-08-15T18:01:57.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.041+0000 7f1d088c0700 1 Processor -- start 2024-08-15T18:01:57.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.041+0000 7f1d088c0700 1 -- start start 2024-08-15T18:01:57.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1d040ce640 0x7f1d040b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1d040cc400 0x7f1d040c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d040c5bb0 con 0x7f1d040cea10 2024-08-15T18:01:57.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d040c4fa0 con 0x7f1d040cc400 2024-08-15T18:01:57.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d040c5280 con 0x7f1d040ce640 2024-08-15T18:01:57.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d03fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1d040cc400 0x7f1d040c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d02ffd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1d040cea10 0x7f1d040cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41318/0 (socket says 172.21.15.38:41318) 2024-08-15T18:01:57.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d02ffd700 1 -- 172.21.15.38:0/3057111817 learned_addr learned my addr 172.21.15.38:0/3057111817 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d037fe700 1 --2- 172.21.15.38:0/3057111817 >> v2:172.21.15.104:3300/0 conn(0x7f1d040ce640 0x7f1d040b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d027fc700 1 -- 172.21.15.38:0/3057111817 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455600298 0 0) 0x7f1d040c5bb0 con 0x7f1d040cea10 2024-08-15T18:01:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d027fc700 1 -- 172.21.15.38:0/3057111817 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cec003570 con 0x7f1d040cea10 2024-08-15T18:01:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d027fc700 1 -- 172.21.15.38:0/3057111817 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2644344822 0 0) 0x7f1cec003570 con 0x7f1d040cea10 2024-08-15T18:01:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d03fff700 1 -- 172.21.15.38:0/3057111817 >> v2:172.21.15.104:3300/0 conn(0x7f1d040ce640 msgr2=0x7f1d040b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d03fff700 1 --2- 172.21.15.38:0/3057111817 >> v2:172.21.15.104:3300/0 conn(0x7f1d040ce640 0x7f1d040b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d03fff700 1 -- 172.21.15.38:0/3057111817 >> v1:172.21.15.38:6789/0 conn(0x7f1d040cea10 legacy=0x7f1d040cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d03fff700 1 -- 172.21.15.38:0/3057111817 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d040c5560 con 0x7f1d040cc400 2024-08-15T18:01:57.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d03fff700 1 --2- 172.21.15.38:0/3057111817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d040cc400 0x7f1d040c6040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1d040d0030 tx=0x7f1cfc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316dfa4390db9803 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d027fc700 1 -- 172.21.15.38:0/3057111817 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00e070 con 0x7f1d040cc400 2024-08-15T18:01:57.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d027fc700 1 -- 172.21.15.38:0/3057111817 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc00b7a0 con 0x7f1d040cc400 2024-08-15T18:01:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d027fc700 1 -- 172.21.15.38:0/3057111817 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4001470 con 0x7f1d040cc400 2024-08-15T18:01:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 -- 172.21.15.38:0/3057111817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d040cc400 msgr2=0x7f1d040c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 --2- 172.21.15.38:0/3057111817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d040cc400 0x7f1d040c6040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1d040d0030 tx=0x7f1cfc009780 comp rx=0 tx=0).stop 2024-08-15T18:01:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 -- 172.21.15.38:0/3057111817 shutdown_connections 2024-08-15T18:01:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 --2- 172.21.15.38:0/3057111817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d040cc400 0x7f1d040c6040 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 --2- 172.21.15.38:0/3057111817 >> v2:172.21.15.104:3300/0 conn(0x7f1d040ce640 0x7f1d040b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 -- 172.21.15.38:0/3057111817 >> 172.21.15.38:0/3057111817 conn(0x7f1d04099410 msgr2=0x7f1d040675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 -- 172.21.15.38:0/3057111817 shutdown_connections 2024-08-15T18:01:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.045+0000 7f1d088c0700 1 -- 172.21.15.38:0/3057111817 wait complete. 2024-08-15T18:01:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d088c0700 1 Processor -- start 2024-08-15T18:01:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d088c0700 1 -- start start 2024-08-15T18:01:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d088c0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d040cc400 0x7f1d04169f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d088c0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d040ce640 0x7f1d04068880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d088c0700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1d040cea10 0x7f1d04168d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d088c0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04099d80 con 0x7f1d040ce640 2024-08-15T18:01:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d088c0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04169870 con 0x7f1d040cc400 2024-08-15T18:01:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d088c0700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04169b50 con 0x7f1d040cea10 2024-08-15T18:01:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d02ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d040ce640 0x7f1d04068880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d02ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d040ce640 0x7f1d04068880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37972/0 (socket says 172.21.15.38:37972) 2024-08-15T18:01:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d02ffd700 1 -- 172.21.15.38:0/3013487777 learned_addr learned my addr 172.21.15.38:0/3013487777 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d037fe700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d040cc400 0x7f1d04169f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.049+0000 7f1d03fff700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1d040cea10 0x7f1d04168d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1d02ffd700 1 -- 172.21.15.38:0/3013487777 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1d040cea10 msgr2=0x7f1d04168d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1d02ffd700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1d040cea10 0x7f1d04168d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1d02ffd700 1 -- 172.21.15.38:0/3013487777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d040cc400 msgr2=0x7f1d04169f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1d02ffd700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d040cc400 0x7f1d04169f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1d02ffd700 1 -- 172.21.15.38:0/3013487777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc009a20 con 0x7f1d040ce640 2024-08-15T18:01:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1d02ffd700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d040ce640 0x7f1d04068880 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400cb90 tx=0x7f1cf4001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1ceb7fe700 1 -- 172.21.15.38:0/3013487777 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4007150 con 0x7f1d040ce640 2024-08-15T18:01:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1ceb7fe700 1 -- 172.21.15.38:0/3013487777 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf40072d0 con 0x7f1d040ce640 2024-08-15T18:01:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1ceb7fe700 1 -- 172.21.15.38:0/3013487777 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4007690 con 0x7f1d040ce640 2024-08-15T18:01:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1d088c0700 1 -- 172.21.15.38:0/3013487777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d04099fd0 con 0x7f1d040ce640 2024-08-15T18:01:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1d088c0700 1 -- 172.21.15.38:0/3013487777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d040c9d60 con 0x7f1d040ce640 2024-08-15T18:01:57.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1d088c0700 1 -- 172.21.15.38:0/3013487777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d040644b0 con 0x7f1d040ce640 2024-08-15T18:01:57.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.053+0000 7f1ceb7fe700 1 -- 172.21.15.38:0/3013487777 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1cf4004a40 con 0x7f1d040ce640 2024-08-15T18:01:57.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.057+0000 7f1ceb7fe700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1cf0060860 0x7f1cf0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.057+0000 7f1ceb7fe700 1 -- 172.21.15.38:0/3013487777 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f1cf407c800 con 0x7f1d040ce640 2024-08-15T18:01:57.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.057+0000 7f1ceb7fe700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] conn(0x7f1cf0066750 0x7f1cf0068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.057+0000 7f1ceb7fe700 1 -- 172.21.15.38:0/3013487777 --> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1cf0069210 con 0x7f1cf0066750 2024-08-15T18:01:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.057+0000 7f1ceb7fe700 1 -- 172.21.15.38:0/3013487777 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf407cbd0 con 0x7f1d040ce640 2024-08-15T18:01:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.057+0000 7f1d037fe700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1cf0060860 0x7f1cf0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.057+0000 7f1d03fff700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] conn(0x7f1cf0066750 0x7f1cf0068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.057+0000 7f1d03fff700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] conn(0x7f1cf0066750 0x7f1cf0068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.057+0000 7f1d037fe700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1cf0060860 0x7f1cf0062d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d040c9330 tx=0x7f1cf80091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.057+0000 7f1ceb7fe700 1 -- 172.21.15.38:0/3013487777 <== osd.3 v2:172.21.15.38:6800/24120 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1cf0069210 con 0x7f1cf0066750 2024-08-15T18:01:57.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.073+0000 7f1d088c0700 1 -- 172.21.15.38:0/3013487777 --> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1d0416a450 con 0x7f1cf0066750 2024-08-15T18:01:57.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1ceb7fe700 1 -- 172.21.15.38:0/3013487777 <== osd.3 v2:172.21.15.38:6800/24120 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1d0416a450 con 0x7f1cf0066750 2024-08-15T18:01:57.080 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T18:01:57.080 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T18:01:57.080 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T18:01:57.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1d088c0700 1 -- 172.21.15.38:0/3013487777 >> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] conn(0x7f1cf0066750 msgr2=0x7f1cf0068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1d088c0700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.38:6800/24120,v1:172.21.15.38:6801/24120] conn(0x7f1cf0066750 0x7f1cf0068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1d088c0700 1 -- 172.21.15.38:0/3013487777 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1cf0060860 msgr2=0x7f1cf0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1d088c0700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f1cf0060860 0x7f1cf0062d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d040c9330 tx=0x7f1cf80091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1d088c0700 1 -- 172.21.15.38:0/3013487777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d040ce640 msgr2=0x7f1d04068880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1d088c0700 1 --2- 172.21.15.38:0/3013487777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d040ce640 0x7f1d04068880 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400cb90 tx=0x7f1cf4001ab0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1d03fff700 1 -- 172.21.15.38:0/3013487777 reap_dead start 2024-08-15T18:01:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1d088c0700 1 -- 172.21.15.38:0/3013487777 shutdown_connections 2024-08-15T18:01:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1d088c0700 1 -- 172.21.15.38:0/3013487777 >> 172.21.15.38:0/3013487777 conn(0x7f1d04099410 msgr2=0x7f1d040b5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1d088c0700 1 -- 172.21.15.38:0/3013487777 shutdown_connections 2024-08-15T18:01:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.077+0000 7f1d088c0700 1 -- 172.21.15.38:0/3013487777 wait complete. 2024-08-15T18:01:57.096 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-08-15T18:01:57.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe27085a700 1 Processor -- start 2024-08-15T18:01:57.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe27085a700 1 -- start start 2024-08-15T18:01:57.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe27085a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe26c06e620 0x7fe26c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe27085a700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26c0af2b0 con 0x7fe26c155d20 2024-08-15T18:01:57.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe27085a700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26c0af590 con 0x7fe26c0b0350 2024-08-15T18:01:57.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe27085a700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe26c0b07a0 con 0x7fe26c06e620 2024-08-15T18:01:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe26bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe26c06e620 0x7fe26c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe26bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe26c06e620 0x7fe26c0ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37980/0 (socket says 172.21.15.38:37980) 2024-08-15T18:01:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe26b7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe26c155d20 0x7fe26c0aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41326/0 (socket says 172.21.15.38:41326) 2024-08-15T18:01:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe26bfff700 1 -- 172.21.15.38:0/823959500 learned_addr learned my addr 172.21.15.38:0/823959500 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe26bfff700 1 -- 172.21.15.38:0/823959500 >> v1:172.21.15.104:6789/0 conn(0x7fe26c0b0350 legacy=0x7fe26c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe26bfff700 1 -- 172.21.15.38:0/823959500 >> v1:172.21.15.38:6789/0 conn(0x7fe26c155d20 legacy=0x7fe26c0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe26bfff700 1 -- 172.21.15.38:0/823959500 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c0af870 con 0x7fe26c06e620 2024-08-15T18:01:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe26bfff700 1 --2- 172.21.15.38:0/823959500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe26c06e620 0x7fe26c0ad7c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe264009f20 tx=0x7fe264009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14babb858c17dd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe26a7fc700 1 -- 172.21.15.38:0/823959500 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26400e070 con 0x7fe26c06e620 2024-08-15T18:01:57.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe26a7fc700 1 -- 172.21.15.38:0/823959500 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26400ba80 con 0x7fe26c06e620 2024-08-15T18:01:57.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.205+0000 7fe26a7fc700 1 -- 172.21.15.38:0/823959500 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2640092e0 con 0x7fe26c06e620 2024-08-15T18:01:57.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.209+0000 7fe27085a700 1 -- 172.21.15.38:0/823959500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe26c06e620 msgr2=0x7fe26c0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.209+0000 7fe27085a700 1 --2- 172.21.15.38:0/823959500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe26c06e620 0x7fe26c0ad7c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe264009f20 tx=0x7fe264009840 comp rx=0 tx=0).stop 2024-08-15T18:01:57.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.209+0000 7fe27085a700 1 -- 172.21.15.38:0/823959500 shutdown_connections 2024-08-15T18:01:57.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.209+0000 7fe27085a700 1 --2- 172.21.15.38:0/823959500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe26c06e620 0x7fe26c0ad7c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.209+0000 7fe27085a700 1 -- 172.21.15.38:0/823959500 >> 172.21.15.38:0/823959500 conn(0x7fe26c0678d0 msgr2=0x7fe26c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:57.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.209+0000 7fe27085a700 1 -- 172.21.15.38:0/823959500 shutdown_connections 2024-08-15T18:01:57.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.209+0000 7fe27085a700 1 -- 172.21.15.38:0/823959500 wait complete. 2024-08-15T18:01:57.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe27085a700 1 Processor -- start 2024-08-15T18:01:57.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe27085a700 1 -- start start 2024-08-15T18:01:57.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe27085a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe26c06e620 0x7fe26c14f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe27085a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe26c0b0350 0x7fe26c13e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe27085a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe26c155d20 0x7fe26c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe27085a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c13ed10 con 0x7fe26c0b0350 2024-08-15T18:01:57.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe27085a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c1521b0 con 0x7fe26c06e620 2024-08-15T18:01:57.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe27085a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c0abfe0 con 0x7fe26c155d20 2024-08-15T18:01:57.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe26affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe26c0b0350 0x7fe26c13e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe26bfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe26c155d20 0x7fe26c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe26affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe26c0b0350 0x7fe26c13e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37982/0 (socket says 172.21.15.38:37982) 2024-08-15T18:01:57.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe26affd700 1 -- 172.21.15.38:0/4259176744 learned_addr learned my addr 172.21.15.38:0/4259176744 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:57.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe26b7fe700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe26c06e620 0x7fe26c14f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe26affd700 1 -- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe26c155d20 msgr2=0x7fe26c154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe26affd700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe26c155d20 0x7fe26c154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe26affd700 1 -- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe26c06e620 msgr2=0x7fe26c14f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe26affd700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe26c06e620 0x7fe26c14f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe26affd700 1 -- 172.21.15.38:0/4259176744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe264009ae0 con 0x7fe26c0b0350 2024-08-15T18:01:57.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe26affd700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe26c0b0350 0x7fe26c13e7d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe25400c5f0 tx=0x7fe25400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe2637fe700 1 -- 172.21.15.38:0/4259176744 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254011590 con 0x7fe26c0b0350 2024-08-15T18:01:57.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe2637fe700 1 -- 172.21.15.38:0/4259176744 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe254011710 con 0x7fe26c0b0350 2024-08-15T18:01:57.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe27085a700 1 -- 172.21.15.38:0/4259176744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26c1515a0 con 0x7fe26c0b0350 2024-08-15T18:01:57.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe2637fe700 1 -- 172.21.15.38:0/4259176744 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254011ad0 con 0x7fe26c0b0350 2024-08-15T18:01:57.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.213+0000 7fe27085a700 1 -- 172.21.15.38:0/4259176744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26c151b60 con 0x7fe26c0b0350 2024-08-15T18:01:57.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.217+0000 7fe27085a700 1 -- 172.21.15.38:0/4259176744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26c0643a0 con 0x7fe26c0b0350 2024-08-15T18:01:57.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.217+0000 7fe2637fe700 1 -- 172.21.15.38:0/4259176744 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe25401b3b0 con 0x7fe26c0b0350 2024-08-15T18:01:57.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.217+0000 7fe2637fe700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe24c060860 0x7fe24c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.217+0000 7fe2637fe700 1 -- 172.21.15.38:0/4259176744 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fe25407bde0 con 0x7fe26c0b0350 2024-08-15T18:01:57.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.217+0000 7fe26b7fe700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe24c060860 0x7fe24c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.217+0000 7fe26b7fe700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe24c060860 0x7fe24c062d10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe25c004fd0 tx=0x7fe25c007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.225+0000 7fe2637fe700 1 -- 172.21.15.38:0/4259176744 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe25404c430 con 0x7fe26c0b0350 2024-08-15T18:01:57.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.389+0000 7fe27085a700 1 -- 172.21.15.38:0/4259176744 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe26c1522f0 con 0x7fe24c060860 2024-08-15T18:01:57.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.389+0000 7fe2637fe700 1 -- 172.21.15.38:0/4259176744 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe26c1522f0 con 0x7fe24c060860 2024-08-15T18:01:57.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 -- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe24c060860 msgr2=0x7fe24c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe24c060860 0x7fe24c062d10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe25c004fd0 tx=0x7fe25c007eb0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 -- 172.21.15.38:0/4259176744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe26c0b0350 msgr2=0x7fe26c13e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe26c0b0350 0x7fe26c13e7d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe25400c5f0 tx=0x7fe25400c900 comp rx=0 tx=0).stop 2024-08-15T18:01:57.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 -- 172.21.15.38:0/4259176744 shutdown_connections 2024-08-15T18:01:57.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fe24c060860 0x7fe24c062d10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe26c155d20 0x7fe26c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe26c0b0350 0x7fe26c13e7d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 --2- 172.21.15.38:0/4259176744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe26c06e620 0x7fe26c14f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 -- 172.21.15.38:0/4259176744 >> 172.21.15.38:0/4259176744 conn(0x7fe26c0678d0 msgr2=0x7fe26c148830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:57.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 -- 172.21.15.38:0/4259176744 shutdown_connections 2024-08-15T18:01:57.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.393+0000 7fe27085a700 1 -- 172.21.15.38:0/4259176744 wait complete. 2024-08-15T18:01:57.399 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 3 to deep-scrub 2024-08-15T18:01:57.413 INFO:tasks.ceph:Scrubbing osd.4 2024-08-15T18:01:57.413 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-08-15T18:01:57.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e759e700 1 Processor -- start 2024-08-15T18:01:57.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e759e700 1 -- start start 2024-08-15T18:01:57.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f12e8155d70 0x7f12e80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e759e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f12e8069f00 0x7f12e80b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e759e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e80b1560 con 0x7f12e80b20f0 2024-08-15T18:01:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e759e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e8069a70 con 0x7f12e8069f00 2024-08-15T18:01:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e80cb910 con 0x7f12e8155d70 2024-08-15T18:01:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e5d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f12e8069f00 0x7f12e80b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e5d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f12e8069f00 0x7f12e80b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37996/0 (socket says 172.21.15.38:37996) 2024-08-15T18:01:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e5d9b700 1 -- 172.21.15.38:0/3441909527 learned_addr learned my addr 172.21.15.38:0/3441909527 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e659c700 1 --2- 172.21.15.38:0/3441909527 >> v2:172.21.15.104:3300/0 conn(0x7f12e8155d70 0x7f12e80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e5d9b700 1 -- 172.21.15.38:0/3441909527 >> v2:172.21.15.104:3300/0 conn(0x7f12e8155d70 msgr2=0x7f12e80687a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e5d9b700 1 --2- 172.21.15.38:0/3441909527 >> v2:172.21.15.104:3300/0 conn(0x7f12e8155d70 0x7f12e80687a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e5d9b700 1 -- 172.21.15.38:0/3441909527 >> v1:172.21.15.104:6789/0 conn(0x7f12e80b20f0 legacy=0x7f12e8159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e5d9b700 1 -- 172.21.15.38:0/3441909527 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e8068e60 con 0x7f12e8069f00 2024-08-15T18:01:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e5d9b700 1 --2- 172.21.15.38:0/3441909527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e8069f00 0x7f12e80b1020 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f12e806e070 tx=0x7f12dc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1d3a0dd52ac032 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e559a700 1 -- 172.21.15.38:0/3441909527 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc00dba0 con 0x7f12e8069f00 2024-08-15T18:01:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e559a700 1 -- 172.21.15.38:0/3441909527 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc00dd00 con 0x7f12e8069f00 2024-08-15T18:01:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.529+0000 7f12e559a700 1 -- 172.21.15.38:0/3441909527 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc0196d0 con 0x7f12e8069f00 2024-08-15T18:01:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.533+0000 7f12e759e700 1 -- 172.21.15.38:0/3441909527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e8069f00 msgr2=0x7f12e80b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.533+0000 7f12e759e700 1 --2- 172.21.15.38:0/3441909527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e8069f00 0x7f12e80b1020 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f12e806e070 tx=0x7f12dc00b4f0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.533+0000 7f12e759e700 1 -- 172.21.15.38:0/3441909527 shutdown_connections 2024-08-15T18:01:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.533+0000 7f12e759e700 1 --2- 172.21.15.38:0/3441909527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e8069f00 0x7f12e80b1020 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.533+0000 7f12e759e700 1 --2- 172.21.15.38:0/3441909527 >> v2:172.21.15.104:3300/0 conn(0x7f12e8155d70 0x7f12e80687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.533+0000 7f12e759e700 1 -- 172.21.15.38:0/3441909527 >> 172.21.15.38:0/3441909527 conn(0x7f12e806a700 msgr2=0x7f12e80aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:57.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.533+0000 7f12e759e700 1 -- 172.21.15.38:0/3441909527 shutdown_connections 2024-08-15T18:01:57.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.533+0000 7f12e759e700 1 -- 172.21.15.38:0/3441909527 wait complete. 2024-08-15T18:01:57.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.533+0000 7f12e759e700 1 Processor -- start 2024-08-15T18:01:57.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.533+0000 7f12e759e700 1 -- start start 2024-08-15T18:01:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.533+0000 7f12e759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12e8069f00 0x7f12e8146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e759e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12e80b20f0 0x7f12e80ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e759e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e8155d70 0x7f12e80ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e759e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e8164e00 con 0x7f12e8155d70 2024-08-15T18:01:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e81650e0 con 0x7f12e8069f00 2024-08-15T18:01:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e759e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e81653c0 con 0x7f12e80b20f0 2024-08-15T18:01:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e6d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e8155d70 0x7f12e80ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12e8069f00 0x7f12e8146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e5d9b700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12e80b20f0 0x7f12e80ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e6d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e8155d70 0x7f12e80ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37998/0 (socket says 172.21.15.38:37998) 2024-08-15T18:01:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12e8069f00 0x7f12e8146330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:58864/0 (socket says 172.21.15.38:58864) 2024-08-15T18:01:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e6d9d700 1 -- 172.21.15.38:0/54340489 learned_addr learned my addr 172.21.15.38:0/54340489 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e6d9d700 1 -- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12e80b20f0 msgr2=0x7f12e80ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e6d9d700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12e80b20f0 0x7f12e80ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e6d9d700 1 -- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12e8069f00 msgr2=0x7f12e8146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e6d9d700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12e8069f00 0x7f12e8146330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e6d9d700 1 -- 172.21.15.38:0/54340489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc00b790 con 0x7f12e8155d70 2024-08-15T18:01:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e6d9d700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e8155d70 0x7f12e80ae9b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f12e000c8e0 tx=0x7f12e0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12d67fc700 1 -- 172.21.15.38:0/54340489 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e0012630 con 0x7f12e8155d70 2024-08-15T18:01:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12d67fc700 1 -- 172.21.15.38:0/54340489 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e0012790 con 0x7f12e8155d70 2024-08-15T18:01:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e759e700 1 -- 172.21.15.38:0/54340489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e81656a0 con 0x7f12e8155d70 2024-08-15T18:01:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12d67fc700 1 -- 172.21.15.38:0/54340489 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e0012b50 con 0x7f12e8155d70 2024-08-15T18:01:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e759e700 1 -- 172.21.15.38:0/54340489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e8146a10 con 0x7f12e8155d70 2024-08-15T18:01:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.537+0000 7f12e759e700 1 -- 172.21.15.38:0/54340489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f12e80644b0 con 0x7f12e8155d70 2024-08-15T18:01:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.541+0000 7f12d67fc700 1 -- 172.21.15.38:0/54340489 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f12e001a3a0 con 0x7f12e8155d70 2024-08-15T18:01:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.541+0000 7f12d67fc700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f12c8060860 0x7f12c8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.541+0000 7f12d67fc700 1 -- 172.21.15.38:0/54340489 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f12e007c8f0 con 0x7f12e8155d70 2024-08-15T18:01:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.541+0000 7f12e659c700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f12c8060860 0x7f12c8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.541+0000 7f12d67fc700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] conn(0x7f12c8066750 0x7f12c8068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.541+0000 7f12d67fc700 1 -- 172.21.15.38:0/54340489 --> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12c8069210 con 0x7f12c8066750 2024-08-15T18:01:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.541+0000 7f12d67fc700 1 -- 172.21.15.38:0/54340489 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12e0081240 con 0x7f12e8155d70 2024-08-15T18:01:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.541+0000 7f12e5d9b700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] conn(0x7f12c8066750 0x7f12c8068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.541+0000 7f12e659c700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f12c8060860 0x7f12c8062d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f12e814a880 tx=0x7f12d80091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.541+0000 7f12e5d9b700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] conn(0x7f12c8066750 0x7f12c8068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.545+0000 7f12d67fc700 1 -- 172.21.15.38:0/54340489 <== osd.4 v2:172.21.15.104:6824/24496 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f12c8069210 con 0x7f12c8066750 2024-08-15T18:01:57.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.561+0000 7f12e759e700 1 -- 172.21.15.38:0/54340489 --> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f12e80b1560 con 0x7f12c8066750 2024-08-15T18:01:57.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.561+0000 7f12d67fc700 1 -- 172.21.15.38:0/54340489 <== osd.4 v2:172.21.15.104:6824/24496 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f12e80b1560 con 0x7f12c8066750 2024-08-15T18:01:57.568 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T18:01:57.568 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T18:01:57.568 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T18:01:57.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.565+0000 7f12e759e700 1 -- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] conn(0x7f12c8066750 msgr2=0x7f12c8068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.565+0000 7f12e759e700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:6824/24496,v1:172.21.15.104:6825/24496] conn(0x7f12c8066750 0x7f12c8068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.565+0000 7f12e759e700 1 -- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f12c8060860 msgr2=0x7f12c8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.565+0000 7f12e759e700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f12c8060860 0x7f12c8062d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f12e814a880 tx=0x7f12d80091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.565+0000 7f12e759e700 1 -- 172.21.15.38:0/54340489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e8155d70 msgr2=0x7f12e80ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.565+0000 7f12e759e700 1 --2- 172.21.15.38:0/54340489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e8155d70 0x7f12e80ae9b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f12e000c8e0 tx=0x7f12e0001710 comp rx=0 tx=0).stop 2024-08-15T18:01:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.565+0000 7f12e6d9d700 1 -- 172.21.15.38:0/54340489 reap_dead start 2024-08-15T18:01:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.565+0000 7f12e759e700 1 -- 172.21.15.38:0/54340489 shutdown_connections 2024-08-15T18:01:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.565+0000 7f12e759e700 1 -- 172.21.15.38:0/54340489 >> 172.21.15.38:0/54340489 conn(0x7f12e806a700 msgr2=0x7f12e80acec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.565+0000 7f12e759e700 1 -- 172.21.15.38:0/54340489 shutdown_connections 2024-08-15T18:01:57.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.565+0000 7f12e759e700 1 -- 172.21.15.38:0/54340489 wait complete. 2024-08-15T18:01:57.585 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-08-15T18:01:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.689+0000 7f2ffe795700 1 Processor -- start 2024-08-15T18:01:57.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.689+0000 7f2ffe795700 1 -- start start 2024-08-15T18:01:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.689+0000 7f2ffe795700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2ff8157070 0x7f2ff80b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.689+0000 7f2ffe795700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2ff8157440 0x7f2ff815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.689+0000 7f2ffe795700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff80b1ae0 con 0x7f2ff815b980 2024-08-15T18:01:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.689+0000 7f2ffe795700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff80ad090 con 0x7f2ff8157440 2024-08-15T18:01:57.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.689+0000 7f2ffe795700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff80b0ed0 con 0x7f2ff8157070 2024-08-15T18:01:57.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.689+0000 7f2ffcf92700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2ff8157440 0x7f2ff815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.689+0000 7f2ffcf92700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2ff8157440 0x7f2ff815ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38010/0 (socket says 172.21.15.38:38010) 2024-08-15T18:01:57.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.689+0000 7f2ffcf92700 1 -- 172.21.15.38:0/2452304616 learned_addr learned my addr 172.21.15.38:0/2452304616 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:57.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.689+0000 7f2ffd793700 1 --2- 172.21.15.38:0/2452304616 >> v2:172.21.15.104:3300/0 conn(0x7f2ff8157070 0x7f2ff80b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffcf92700 1 -- 172.21.15.38:0/2452304616 >> v2:172.21.15.104:3300/0 conn(0x7f2ff8157070 msgr2=0x7f2ff80b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:57.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffcf92700 1 --2- 172.21.15.38:0/2452304616 >> v2:172.21.15.104:3300/0 conn(0x7f2ff8157070 0x7f2ff80b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffcf92700 1 -- 172.21.15.38:0/2452304616 >> v1:172.21.15.104:6789/0 conn(0x7f2ff815b980 legacy=0x7f2ff8066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffcf92700 1 -- 172.21.15.38:0/2452304616 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff80b11b0 con 0x7f2ff8157440 2024-08-15T18:01:57.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffcf92700 1 --2- 172.21.15.38:0/2452304616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ff8157440 0x7f2ff815ad40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2ff806cef0 tx=0x7f2ff4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf47fa243fd21f83 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2feffff700 1 -- 172.21.15.38:0/2452304616 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff400e070 con 0x7f2ff8157440 2024-08-15T18:01:57.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2feffff700 1 -- 172.21.15.38:0/2452304616 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff400ba20 con 0x7f2ff8157440 2024-08-15T18:01:57.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2feffff700 1 -- 172.21.15.38:0/2452304616 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff40092e0 con 0x7f2ff8157440 2024-08-15T18:01:57.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 -- 172.21.15.38:0/2452304616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ff8157440 msgr2=0x7f2ff815ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 --2- 172.21.15.38:0/2452304616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ff8157440 0x7f2ff815ad40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2ff806cef0 tx=0x7f2ff4009840 comp rx=0 tx=0).stop 2024-08-15T18:01:57.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 -- 172.21.15.38:0/2452304616 shutdown_connections 2024-08-15T18:01:57.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 --2- 172.21.15.38:0/2452304616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ff8157440 0x7f2ff815ad40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 --2- 172.21.15.38:0/2452304616 >> v2:172.21.15.104:3300/0 conn(0x7f2ff8157070 0x7f2ff80b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 -- 172.21.15.38:0/2452304616 >> 172.21.15.38:0/2452304616 conn(0x7f2ff80ccb70 msgr2=0x7f2ff80a8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:57.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 -- 172.21.15.38:0/2452304616 shutdown_connections 2024-08-15T18:01:57.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 -- 172.21.15.38:0/2452304616 wait complete. 2024-08-15T18:01:57.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 Processor -- start 2024-08-15T18:01:57.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 -- start start 2024-08-15T18:01:57.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ff8157070 0x7f2ff8098950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ff8157440 0x7f2ff814baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2ff815b980 0x7f2ff813fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff814bfe0 con 0x7f2ff8157070 2024-08-15T18:01:57.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff814d320 con 0x7f2ff8157440 2024-08-15T18:01:57.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffe795700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff814c280 con 0x7f2ff815b980 2024-08-15T18:01:57.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffd793700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ff8157070 0x7f2ff8098950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffd793700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ff8157070 0x7f2ff8098950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38026/0 (socket says 172.21.15.38:38026) 2024-08-15T18:01:57.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.693+0000 7f2ffd793700 1 -- 172.21.15.38:0/389529250 learned_addr learned my addr 172.21.15.38:0/389529250 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:57.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffcf92700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ff8157440 0x7f2ff814baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffdf94700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2ff815b980 0x7f2ff813fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffd793700 1 -- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2ff815b980 msgr2=0x7f2ff813fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffd793700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2ff815b980 0x7f2ff813fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffd793700 1 -- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ff8157440 msgr2=0x7f2ff814baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffd793700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ff8157440 0x7f2ff814baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffd793700 1 -- 172.21.15.38:0/389529250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4009ae0 con 0x7f2ff8157070 2024-08-15T18:01:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffcf92700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ff8157440 0x7f2ff814baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffd793700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ff8157070 0x7f2ff8098950 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800e930 tx=0x7f2fe80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2fed7fa700 1 -- 172.21.15.38:0/389529250 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800eed0 con 0x7f2ff8157070 2024-08-15T18:01:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2fed7fa700 1 -- 172.21.15.38:0/389529250 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe800c850 con 0x7f2ff8157070 2024-08-15T18:01:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffe795700 1 -- 172.21.15.38:0/389529250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff814c560 con 0x7f2ff8157070 2024-08-15T18:01:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2fed7fa700 1 -- 172.21.15.38:0/389529250 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800cbf0 con 0x7f2ff8157070 2024-08-15T18:01:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffe795700 1 -- 172.21.15.38:0/389529250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff814cb20 con 0x7f2ff8157070 2024-08-15T18:01:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2ffe795700 1 -- 172.21.15.38:0/389529250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff80638f0 con 0x7f2ff8157070 2024-08-15T18:01:57.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.697+0000 7f2fed7fa700 1 -- 172.21.15.38:0/389529250 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2fe8013470 con 0x7f2ff8157070 2024-08-15T18:01:57.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.701+0000 7f2fed7fa700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2fe4060860 0x7f2fe4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:57.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.701+0000 7f2fed7fa700 1 -- 172.21.15.38:0/389529250 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f2fe807cbb0 con 0x7f2ff8157070 2024-08-15T18:01:57.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.701+0000 7f2ffcf92700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2fe4060860 0x7f2fe4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:57.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.701+0000 7f2ffcf92700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2fe4060860 0x7f2fe4062d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2ff806d4d0 tx=0x7f2ff4018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:57.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.709+0000 7f2fed7fa700 1 -- 172.21.15.38:0/389529250 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2fe804d200 con 0x7f2ff8157070 2024-08-15T18:01:57.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.869+0000 7f2ffe795700 1 -- 172.21.15.38:0/389529250 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ff8071560 con 0x7f2fe4060860 2024-08-15T18:01:57.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.869+0000 7f2fed7fa700 1 -- 172.21.15.38:0/389529250 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2ff8071560 con 0x7f2fe4060860 2024-08-15T18:01:57.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 -- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2fe4060860 msgr2=0x7f2fe4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2fe4060860 0x7f2fe4062d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2ff806d4d0 tx=0x7f2ff4018000 comp rx=0 tx=0).stop 2024-08-15T18:01:57.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 -- 172.21.15.38:0/389529250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ff8157070 msgr2=0x7f2ff8098950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ff8157070 0x7f2ff8098950 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800e930 tx=0x7f2fe80091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 -- 172.21.15.38:0/389529250 shutdown_connections 2024-08-15T18:01:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2fe4060860 0x7f2fe4062d10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2ff815b980 0x7f2ff813fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ff8157440 0x7f2ff814baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 --2- 172.21.15.38:0/389529250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ff8157070 0x7f2ff8098950 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 -- 172.21.15.38:0/389529250 >> 172.21.15.38:0/389529250 conn(0x7f2ff80ccb70 msgr2=0x7f2ff8099130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:57.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 -- 172.21.15.38:0/389529250 shutdown_connections 2024-08-15T18:01:57.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.873+0000 7f2ffe795700 1 -- 172.21.15.38:0/389529250 wait complete. 2024-08-15T18:01:57.879 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 4 to deep-scrub 2024-08-15T18:01:57.889 INFO:tasks.ceph:Scrubbing osd.5 2024-08-15T18:01:57.889 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-08-15T18:01:58.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f542ac98700 1 Processor -- start 2024-08-15T18:01:58.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f542ac98700 1 -- start start 2024-08-15T18:01:58.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f542ac98700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f54240ce640 0x7f54240b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f542ac98700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54240c5bb0 con 0x7f54240cea10 2024-08-15T18:01:58.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f542ac98700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54240c4fa0 con 0x7f54240cc400 2024-08-15T18:01:58.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f542ac98700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54240c5280 con 0x7f54240ce640 2024-08-15T18:01:58.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5429495700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f54240cea10 0x7f54240cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41336/0 (socket says 172.21.15.38:41336) 2024-08-15T18:01:58.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5429495700 1 -- 172.21.15.38:0/776167089 learned_addr learned my addr 172.21.15.38:0/776167089 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:58.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5429c96700 1 --2- 172.21.15.38:0/776167089 >> v2:172.21.15.104:3300/0 conn(0x7f54240ce640 0x7f54240b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227352214 0 0) 0x7f54240c5bb0 con 0x7f54240cea10 2024-08-15T18:01:58.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5410003570 con 0x7f54240cea10 2024-08-15T18:01:58.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859167363 0 0) 0x7f54240c4fa0 con 0x7f54240cc400 2024-08-15T18:01:58.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54240c5bb0 con 0x7f54240cc400 2024-08-15T18:01:58.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2320988161 0 0) 0x7f5410003570 con 0x7f54240cea10 2024-08-15T18:01:58.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 >> v2:172.21.15.104:3300/0 conn(0x7f54240ce640 msgr2=0x7f54240b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:58.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 --2- 172.21.15.38:0/776167089 >> v2:172.21.15.104:3300/0 conn(0x7f54240ce640 0x7f54240b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 >> v1:172.21.15.104:6789/0 conn(0x7f54240cc400 legacy=0x7f54240c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54240c5560 con 0x7f54240cea10 2024-08-15T18:01:58.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1318163811 0 0) 0x7f5414001470 con 0x7f54240cea10 2024-08-15T18:01:58.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 >> v1:172.21.15.38:6789/0 conn(0x7f54240cea10 legacy=0x7f54240cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 --2- 172.21.15.38:0/776167089 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5410003c40 0x7f5410004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 --2- 172.21.15.38:0/776167089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5410004810 0x7f5410004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 --2- 172.21.15.38:0/776167089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5410005210 0x7f541000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54240c5280 con 0x7f5410004810 2024-08-15T18:01:58.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541000c1b0 con 0x7f5410005210 2024-08-15T18:01:58.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541000c370 con 0x7f5410003c40 2024-08-15T18:01:58.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5429495700 1 --2- 172.21.15.38:0/776167089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5410004810 0x7f5410004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f542a497700 1 --2- 172.21.15.38:0/776167089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5410005210 0x7f541000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:57.997+0000 7f5429c96700 1 --2- 172.21.15.38:0/776167089 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5410003c40 0x7f5410004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f5429495700 1 -- 172.21.15.38:0/776167089 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5410003c40 msgr2=0x7f5410004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:58.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f5429495700 1 --2- 172.21.15.38:0/776167089 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5410003c40 0x7f5410004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f5429495700 1 -- 172.21.15.38:0/776167089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5410005210 msgr2=0x7f541000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:58.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f5429495700 1 --2- 172.21.15.38:0/776167089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5410005210 0x7f541000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f542a497700 1 -- 172.21.15.38:0/776167089 reap_dead start 2024-08-15T18:01:58.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f5429495700 1 -- 172.21.15.38:0/776167089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5410003570 con 0x7f5410004810 2024-08-15T18:01:58.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f5429495700 1 --2- 172.21.15.38:0/776167089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5410004810 0x7f5410004c80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f541400c1e0 tx=0x7f5414001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20941a04342035df server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541400cbe0 con 0x7f5410004810 2024-08-15T18:01:58.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541400cd40 con 0x7f5410004810 2024-08-15T18:01:58.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f5428c94700 1 -- 172.21.15.38:0/776167089 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414005b70 con 0x7f5410004810 2024-08-15T18:01:58.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f542ac98700 1 -- 172.21.15.38:0/776167089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5410004810 msgr2=0x7f5410004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f542ac98700 1 --2- 172.21.15.38:0/776167089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5410004810 0x7f5410004c80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f541400c1e0 tx=0x7f5414001d40 comp rx=0 tx=0).stop 2024-08-15T18:01:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f542ac98700 1 -- 172.21.15.38:0/776167089 shutdown_connections 2024-08-15T18:01:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f542ac98700 1 --2- 172.21.15.38:0/776167089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5410004810 0x7f5410004c80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f542ac98700 1 -- 172.21.15.38:0/776167089 >> 172.21.15.38:0/776167089 conn(0x7f5424099410 msgr2=0x7f54240675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f542ac98700 1 -- 172.21.15.38:0/776167089 shutdown_connections 2024-08-15T18:01:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f542ac98700 1 -- 172.21.15.38:0/776167089 wait complete. 2024-08-15T18:01:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f542ac98700 1 Processor -- start 2024-08-15T18:01:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.001+0000 7f542ac98700 1 -- start start 2024-08-15T18:01:58.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542ac98700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5410004810 0x7f5424155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542ac98700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5424157950 0x7f5424157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542ac98700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54240c8800 0x7f54240c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542ac98700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54240c9570 con 0x7f54240c8800 2024-08-15T18:01:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542ac98700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5424158300 con 0x7f5424157950 2024-08-15T18:01:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542ac98700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542416a320 con 0x7f5410004810 2024-08-15T18:01:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542a497700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54240c8800 0x7f54240c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542a497700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54240c8800 0x7f54240c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38030/0 (socket says 172.21.15.38:38030) 2024-08-15T18:01:58.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542a497700 1 -- 172.21.15.38:0/3399465369 learned_addr learned my addr 172.21.15.38:0/3399465369 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:58.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f5429c96700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5410004810 0x7f5424155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f5429495700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5424157950 0x7f5424157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542a497700 1 -- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5410004810 msgr2=0x7f5424155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542a497700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5410004810 0x7f5424155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542a497700 1 -- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5424157950 msgr2=0x7f5424157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542a497700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5424157950 0x7f5424157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542a497700 1 -- 172.21.15.38:0/3399465369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424169e90 con 0x7f54240c8800 2024-08-15T18:01:58.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542a497700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54240c8800 0x7f54240c8ca0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f54240cc860 tx=0x7f541c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f5419ffb700 1 -- 172.21.15.38:0/3399465369 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c006db0 con 0x7f54240c8800 2024-08-15T18:01:58.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f5419ffb700 1 -- 172.21.15.38:0/3399465369 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c00c570 con 0x7f54240c8800 2024-08-15T18:01:58.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f5419ffb700 1 -- 172.21.15.38:0/3399465369 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c00c930 con 0x7f54240c8800 2024-08-15T18:01:58.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542ac98700 1 -- 172.21.15.38:0/3399465369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5424169300 con 0x7f54240c8800 2024-08-15T18:01:58.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542ac98700 1 -- 172.21.15.38:0/3399465369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5424169560 con 0x7f54240c8800 2024-08-15T18:01:58.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.005+0000 7f542ac98700 1 -- 172.21.15.38:0/3399465369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f54240644b0 con 0x7f54240c8800 2024-08-15T18:01:58.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.009+0000 7f5419ffb700 1 -- 172.21.15.38:0/3399465369 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f541c00cce0 con 0x7f54240c8800 2024-08-15T18:01:58.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.009+0000 7f5419ffb700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5408060890 0x7f5408062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.009+0000 7f5419ffb700 1 -- 172.21.15.38:0/3399465369 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f541c07bfe0 con 0x7f54240c8800 2024-08-15T18:01:58.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.009+0000 7f5429c96700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5408060890 0x7f5408062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.009+0000 7f5419ffb700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] conn(0x7f5408066780 0x7f5408068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.009+0000 7f5419ffb700 1 -- 172.21.15.38:0/3399465369 --> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5408069240 con 0x7f5408066780 2024-08-15T18:01:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.009+0000 7f5419ffb700 1 -- 172.21.15.38:0/3399465369 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f541c07e600 con 0x7f54240c8800 2024-08-15T18:01:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.009+0000 7f5429495700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] conn(0x7f5408066780 0x7f5408068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.009+0000 7f5429c96700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5408060890 0x7f5408062d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f54240ca180 tx=0x7f54200091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.009+0000 7f5429495700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] conn(0x7f5408066780 0x7f5408068b90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.013+0000 7f5419ffb700 1 -- 172.21.15.38:0/3399465369 <== osd.5 v2:172.21.15.104:6800/24498 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5408069240 con 0x7f5408066780 2024-08-15T18:01:58.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542ac98700 1 -- 172.21.15.38:0/3399465369 --> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5424070560 con 0x7f5408066780 2024-08-15T18:01:58.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f5419ffb700 1 -- 172.21.15.38:0/3399465369 <== osd.5 v2:172.21.15.104:6800/24498 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5424070560 con 0x7f5408066780 2024-08-15T18:01:58.033 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T18:01:58.034 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T18:01:58.034 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T18:01:58.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542ac98700 1 -- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] conn(0x7f5408066780 msgr2=0x7f5408068b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542ac98700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:6800/24498,v1:172.21.15.104:6801/24498] conn(0x7f5408066780 0x7f5408068b90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542ac98700 1 -- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5408060890 msgr2=0x7f5408062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542ac98700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f5408060890 0x7f5408062d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f54240ca180 tx=0x7f54200091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542ac98700 1 -- 172.21.15.38:0/3399465369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54240c8800 msgr2=0x7f54240c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542ac98700 1 --2- 172.21.15.38:0/3399465369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54240c8800 0x7f54240c8ca0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f54240cc860 tx=0x7f541c0017a0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542a497700 1 -- 172.21.15.38:0/3399465369 reap_dead start 2024-08-15T18:01:58.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542ac98700 1 -- 172.21.15.38:0/3399465369 shutdown_connections 2024-08-15T18:01:58.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542ac98700 1 -- 172.21.15.38:0/3399465369 >> 172.21.15.38:0/3399465369 conn(0x7f5424099410 msgr2=0x7f5424158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:58.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542ac98700 1 -- 172.21.15.38:0/3399465369 shutdown_connections 2024-08-15T18:01:58.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.029+0000 7f542ac98700 1 -- 172.21.15.38:0/3399465369 wait complete. 2024-08-15T18:01:58.047 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-08-15T18:01:58.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.161+0000 7f7980940700 1 Processor -- start 2024-08-15T18:01:58.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.161+0000 7f7980940700 1 -- start start 2024-08-15T18:01:58.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.161+0000 7f7980940700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c0b0350 0x7f797c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.161+0000 7f7980940700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c0af2b0 con 0x7f797c155d20 2024-08-15T18:01:58.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.161+0000 7f7980940700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c0af590 con 0x7f797c06df10 2024-08-15T18:01:58.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.161+0000 7f7980940700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0b07a0 con 0x7f797c0b0350 2024-08-15T18:01:58.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.161+0000 7f797b7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f797c155d20 0x7f797c0aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41338/0 (socket says 172.21.15.38:41338) 2024-08-15T18:01:58.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.161+0000 7f797b7fe700 1 -- 172.21.15.38:0/3210086778 learned_addr learned my addr 172.21.15.38:0/3210086778 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:58.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.161+0000 7f797affd700 1 --2- 172.21.15.38:0/3210086778 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c0b0350 0x7f797c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797a7fc700 1 -- 172.21.15.38:0/3210086778 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28227467 0 0) 0x7f797c0af2b0 con 0x7f797c155d20 2024-08-15T18:01:58.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797a7fc700 1 -- 172.21.15.38:0/3210086778 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7964003570 con 0x7f797c155d20 2024-08-15T18:01:58.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797a7fc700 1 -- 172.21.15.38:0/3210086778 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362400168 0 0) 0x7f797c0af590 con 0x7f797c06df10 2024-08-15T18:01:58.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797a7fc700 1 -- 172.21.15.38:0/3210086778 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f797c0af2b0 con 0x7f797c06df10 2024-08-15T18:01:58.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797a7fc700 1 -- 172.21.15.38:0/3210086778 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1902111399 0 0) 0x7f7964003570 con 0x7f797c155d20 2024-08-15T18:01:58.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797affd700 1 -- 172.21.15.38:0/3210086778 >> v1:172.21.15.104:6789/0 conn(0x7f797c06df10 legacy=0x7f797c1560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797affd700 1 -- 172.21.15.38:0/3210086778 >> v1:172.21.15.38:6789/0 conn(0x7f797c155d20 legacy=0x7f797c0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797affd700 1 -- 172.21.15.38:0/3210086778 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c0af870 con 0x7f797c0b0350 2024-08-15T18:01:58.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797affd700 1 --2- 172.21.15.38:0/3210086778 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c0b0350 0x7f797c0ad7c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f796c009910 tx=0x7f796c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a550d67ca7226c40 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797a7fc700 1 -- 172.21.15.38:0/3210086778 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974001470 con 0x7f797c0b0350 2024-08-15T18:01:58.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797a7fc700 1 -- 172.21.15.38:0/3210086778 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c011420 con 0x7f797c0b0350 2024-08-15T18:01:58.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f797a7fc700 1 -- 172.21.15.38:0/3210086778 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c0117a0 con 0x7f797c0b0350 2024-08-15T18:01:58.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f7980940700 1 -- 172.21.15.38:0/3210086778 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c0b0350 msgr2=0x7f797c0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f7980940700 1 --2- 172.21.15.38:0/3210086778 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c0b0350 0x7f797c0ad7c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f796c009910 tx=0x7f796c0091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f7980940700 1 -- 172.21.15.38:0/3210086778 shutdown_connections 2024-08-15T18:01:58.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f7980940700 1 --2- 172.21.15.38:0/3210086778 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c0b0350 0x7f797c0ad7c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f7980940700 1 -- 172.21.15.38:0/3210086778 >> 172.21.15.38:0/3210086778 conn(0x7f797c0678d0 msgr2=0x7f797c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:58.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f7980940700 1 -- 172.21.15.38:0/3210086778 shutdown_connections 2024-08-15T18:01:58.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f7980940700 1 -- 172.21.15.38:0/3210086778 wait complete. 2024-08-15T18:01:58.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.165+0000 7f7980940700 1 Processor -- start 2024-08-15T18:01:58.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f7980940700 1 -- start start 2024-08-15T18:01:58.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f7980940700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f797c06df10 0x7f797c14a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f7980940700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f797c0b0350 0x7f797c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f7980940700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c155d20 0x7f797c1469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f7980940700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c14bb00 con 0x7f797c0b0350 2024-08-15T18:01:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f7980940700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c14aef0 con 0x7f797c06df10 2024-08-15T18:01:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f7980940700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c14b1d0 con 0x7f797c155d20 2024-08-15T18:01:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f797c0b0350 0x7f797c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f797c0b0350 0x7f797c158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38038/0 (socket says 172.21.15.38:38038) 2024-08-15T18:01:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797affd700 1 -- 172.21.15.38:0/1880055042 learned_addr learned my addr 172.21.15.38:0/1880055042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797b7fe700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f797c06df10 0x7f797c14a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797bfff700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c155d20 0x7f797c1469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797affd700 1 -- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c155d20 msgr2=0x7f797c1469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797affd700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c155d20 0x7f797c1469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797affd700 1 -- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f797c06df10 msgr2=0x7f797c14a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797affd700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f797c06df10 0x7f797c14a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797affd700 1 -- 172.21.15.38:0/1880055042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c009420 con 0x7f797c0b0350 2024-08-15T18:01:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797bfff700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c155d20 0x7f797c1469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797b7fe700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f797c06df10 0x7f797c14a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:01:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f797affd700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f797c0b0350 0x7f797c158b30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f797c06c2d0 tx=0x7f796c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f79637fe700 1 -- 172.21.15.38:0/1880055042 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c01cba0 con 0x7f797c0b0350 2024-08-15T18:01:58.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f7980940700 1 -- 172.21.15.38:0/1880055042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c14b4b0 con 0x7f797c0b0350 2024-08-15T18:01:58.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f79637fe700 1 -- 172.21.15.38:0/1880055042 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c01cd00 con 0x7f797c0b0350 2024-08-15T18:01:58.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f79637fe700 1 -- 172.21.15.38:0/1880055042 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c01a6a0 con 0x7f797c0b0350 2024-08-15T18:01:58.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.169+0000 7f7980940700 1 -- 172.21.15.38:0/1880055042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c14abe0 con 0x7f797c0b0350 2024-08-15T18:01:58.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.173+0000 7f7980940700 1 -- 172.21.15.38:0/1880055042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c14bc70 con 0x7f797c0b0350 2024-08-15T18:01:58.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.173+0000 7f79637fe700 1 -- 172.21.15.38:0/1880055042 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f796c01aa90 con 0x7f797c0b0350 2024-08-15T18:01:58.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.173+0000 7f79637fe700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7968060860 0x7f7968062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.173+0000 7f79637fe700 1 -- 172.21.15.38:0/1880055042 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f796c07e3a0 con 0x7f797c0b0350 2024-08-15T18:01:58.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.173+0000 7f797b7fe700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7968060860 0x7f7968062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.173+0000 7f797b7fe700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7968060860 0x7f7968062d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f79740015a0 tx=0x7f7974000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.185+0000 7f79637fe700 1 -- 172.21.15.38:0/1880055042 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f796c04e9f0 con 0x7f797c0b0350 2024-08-15T18:01:58.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.349+0000 7f7980940700 1 -- 172.21.15.38:0/1880055042 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f797c0643a0 con 0x7f7968060860 2024-08-15T18:01:58.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.349+0000 7f79637fe700 1 -- 172.21.15.38:0/1880055042 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f797c0643a0 con 0x7f7968060860 2024-08-15T18:01:58.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 -- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7968060860 msgr2=0x7f7968062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7968060860 0x7f7968062d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f79740015a0 tx=0x7f7974000bc0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 -- 172.21.15.38:0/1880055042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f797c0b0350 msgr2=0x7f797c158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f797c0b0350 0x7f797c158b30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f797c06c2d0 tx=0x7f796c016000 comp rx=0 tx=0).stop 2024-08-15T18:01:58.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 -- 172.21.15.38:0/1880055042 shutdown_connections 2024-08-15T18:01:58.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f7968060860 0x7f7968062d10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f797c155d20 0x7f797c1469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f797c0b0350 0x7f797c158b30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 --2- 172.21.15.38:0/1880055042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f797c06df10 0x7f797c14a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 -- 172.21.15.38:0/1880055042 >> 172.21.15.38:0/1880055042 conn(0x7f797c0678d0 msgr2=0x7f797c1489f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:58.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 -- 172.21.15.38:0/1880055042 shutdown_connections 2024-08-15T18:01:58.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.353+0000 7f7980940700 1 -- 172.21.15.38:0/1880055042 wait complete. 2024-08-15T18:01:58.359 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 5 to deep-scrub 2024-08-15T18:01:58.372 INFO:tasks.ceph:Scrubbing osd.6 2024-08-15T18:01:58.373 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-08-15T18:01:58.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23ec1ad700 1 Processor -- start 2024-08-15T18:01:58.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23ec1ad700 1 -- start start 2024-08-15T18:01:58.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23ec1ad700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f23e40b3da0 0x7f23e415acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23ec1ad700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f23e40b0f30 0x7f23e40b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23ec1ad700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f23e4159b20 0x7f23e4159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23ec1ad700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e40b18e0 con 0x7f23e40b0f30 2024-08-15T18:01:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23ec1ad700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e415a870 con 0x7f23e40b3da0 2024-08-15T18:01:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23ec1ad700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e40aec40 con 0x7f23e4159b20 2024-08-15T18:01:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23e9748700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f23e40b0f30 0x7f23e40b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23e9748700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f23e40b0f30 0x7f23e40b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38048/0 (socket says 172.21.15.38:38048) 2024-08-15T18:01:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23e9748700 1 -- 172.21.15.38:0/127652813 learned_addr learned my addr 172.21.15.38:0/127652813 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23ea74a700 1 --2- 172.21.15.38:0/127652813 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f23e4159b20 0x7f23e4159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23e9f49700 1 --2- 172.21.15.38:0/127652813 >> v2:172.21.15.104:3300/0 conn(0x7f23e40b3da0 0x7f23e415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23e9748700 1 -- 172.21.15.38:0/127652813 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f23e4159b20 msgr2=0x7f23e4159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.485+0000 7f23e9748700 1 --2- 172.21.15.38:0/127652813 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f23e4159b20 0x7f23e4159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23e9748700 1 -- 172.21.15.38:0/127652813 >> v2:172.21.15.104:3300/0 conn(0x7f23e40b3da0 msgr2=0x7f23e415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23e9748700 1 --2- 172.21.15.38:0/127652813 >> v2:172.21.15.104:3300/0 conn(0x7f23e40b3da0 0x7f23e415acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23e9748700 1 -- 172.21.15.38:0/127652813 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e40aef20 con 0x7f23e40b0f30 2024-08-15T18:01:58.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23e9748700 1 --2- 172.21.15.38:0/127652813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e40b0f30 0x7f23e40b13a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f23e40715e0 tx=0x7f23d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83966fea0501377 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23e8f47700 1 -- 172.21.15.38:0/127652813 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8009e70 con 0x7f23e40b0f30 2024-08-15T18:01:58.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23e8f47700 1 -- 172.21.15.38:0/127652813 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d800b860 con 0x7f23e40b0f30 2024-08-15T18:01:58.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23e8f47700 1 -- 172.21.15.38:0/127652813 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800bc00 con 0x7f23e40b0f30 2024-08-15T18:01:58.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23ec1ad700 1 -- 172.21.15.38:0/127652813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e40b0f30 msgr2=0x7f23e40b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23ec1ad700 1 --2- 172.21.15.38:0/127652813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e40b0f30 0x7f23e40b13a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f23e40715e0 tx=0x7f23d8009250 comp rx=0 tx=0).stop 2024-08-15T18:01:58.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23ec1ad700 1 -- 172.21.15.38:0/127652813 shutdown_connections 2024-08-15T18:01:58.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23ec1ad700 1 --2- 172.21.15.38:0/127652813 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f23e4159b20 0x7f23e4159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23ec1ad700 1 --2- 172.21.15.38:0/127652813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e40b0f30 0x7f23e40b13a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23ec1ad700 1 --2- 172.21.15.38:0/127652813 >> v2:172.21.15.104:3300/0 conn(0x7f23e40b3da0 0x7f23e415acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23ec1ad700 1 -- 172.21.15.38:0/127652813 >> 172.21.15.38:0/127652813 conn(0x7f23e406ce20 msgr2=0x7f23e406d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:58.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23ec1ad700 1 -- 172.21.15.38:0/127652813 shutdown_connections 2024-08-15T18:01:58.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23ec1ad700 1 -- 172.21.15.38:0/127652813 wait complete. 2024-08-15T18:01:58.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.489+0000 7f23ec1ad700 1 Processor -- start 2024-08-15T18:01:58.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23ec1ad700 1 -- start start 2024-08-15T18:01:58.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23ec1ad700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e40b0f30 0x7f23e415c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23ec1ad700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e40b3da0 0x7f23e4069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23ec1ad700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f23e4159b20 0x7f23e41558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23ec1ad700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e4169dc0 con 0x7f23e40b3da0 2024-08-15T18:01:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23ec1ad700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e416a040 con 0x7f23e40b0f30 2024-08-15T18:01:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23ec1ad700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e416a320 con 0x7f23e4159b20 2024-08-15T18:01:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23e9748700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e40b3da0 0x7f23e4069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23e9f49700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e40b0f30 0x7f23e415c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23ea74a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f23e4159b20 0x7f23e41558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23e9748700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e40b3da0 0x7f23e4069860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38052/0 (socket says 172.21.15.38:38052) 2024-08-15T18:01:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23e9748700 1 -- 172.21.15.38:0/156977069 learned_addr learned my addr 172.21.15.38:0/156977069 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23e9748700 1 -- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f23e4159b20 msgr2=0x7f23e41558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23e9748700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f23e4159b20 0x7f23e41558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23e9748700 1 -- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e40b0f30 msgr2=0x7f23e415c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23e9748700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e40b0f30 0x7f23e415c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23e9748700 1 -- 172.21.15.38:0/156977069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d80094f0 con 0x7f23e40b3da0 2024-08-15T18:01:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23ea74a700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f23e4159b20 0x7f23e41558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:01:58.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23e9748700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e40b3da0 0x7f23e4069860 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f23e40682a0 tx=0x7f23d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23d5ffb700 1 -- 172.21.15.38:0/156977069 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800bbe0 con 0x7f23e40b3da0 2024-08-15T18:01:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23d5ffb700 1 -- 172.21.15.38:0/156977069 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d8000c90 con 0x7f23e40b3da0 2024-08-15T18:01:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23ec1ad700 1 -- 172.21.15.38:0/156977069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e416a600 con 0x7f23e40b3da0 2024-08-15T18:01:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23d5ffb700 1 -- 172.21.15.38:0/156977069 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8011650 con 0x7f23e40b3da0 2024-08-15T18:01:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.493+0000 7f23ec1ad700 1 -- 172.21.15.38:0/156977069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e416abc0 con 0x7f23e40b3da0 2024-08-15T18:01:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.497+0000 7f23ec1ad700 1 -- 172.21.15.38:0/156977069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f23e40638f0 con 0x7f23e40b3da0 2024-08-15T18:01:58.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.497+0000 7f23d5ffb700 1 -- 172.21.15.38:0/156977069 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f23d8011a80 con 0x7f23e40b3da0 2024-08-15T18:01:58.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.497+0000 7f23d5ffb700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f23c8060860 0x7f23c8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.497+0000 7f23d5ffb700 1 -- 172.21.15.38:0/156977069 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f23d807e5f0 con 0x7f23e40b3da0 2024-08-15T18:01:58.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.497+0000 7f23e9f49700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f23c8060860 0x7f23c8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.497+0000 7f23d5ffb700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] conn(0x7f23c8066750 0x7f23c8068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.497+0000 7f23d5ffb700 1 -- 172.21.15.38:0/156977069 --> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23c8069210 con 0x7f23c8066750 2024-08-15T18:01:58.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.497+0000 7f23d5ffb700 1 -- 172.21.15.38:0/156977069 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23d807e8e0 con 0x7f23e40b3da0 2024-08-15T18:01:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.501+0000 7f23ea74a700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] conn(0x7f23c8066750 0x7f23c8068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.501+0000 7f23e9f49700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f23c8060860 0x7f23c8062d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f23e0006fd0 tx=0x7f23e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.501+0000 7f23ea74a700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] conn(0x7f23c8066750 0x7f23c8068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.501+0000 7f23d5ffb700 1 -- 172.21.15.38:0/156977069 <== osd.6 v2:172.21.15.104:6808/24497 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f23c8069210 con 0x7f23c8066750 2024-08-15T18:01:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.521+0000 7f23ec1ad700 1 -- 172.21.15.38:0/156977069 --> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f23e406cbe0 con 0x7f23c8066750 2024-08-15T18:01:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.521+0000 7f23d5ffb700 1 -- 172.21.15.38:0/156977069 <== osd.6 v2:172.21.15.104:6808/24497 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f23e406cbe0 con 0x7f23c8066750 2024-08-15T18:01:58.527 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T18:01:58.527 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T18:01:58.527 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T18:01:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.525+0000 7f23ec1ad700 1 -- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] conn(0x7f23c8066750 msgr2=0x7f23c8068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.525+0000 7f23ec1ad700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:6808/24497,v1:172.21.15.104:6809/24497] conn(0x7f23c8066750 0x7f23c8068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.525+0000 7f23ec1ad700 1 -- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f23c8060860 msgr2=0x7f23c8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.525+0000 7f23ec1ad700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f23c8060860 0x7f23c8062d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f23e0006fd0 tx=0x7f23e0008040 comp rx=0 tx=0).stop 2024-08-15T18:01:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.525+0000 7f23ec1ad700 1 -- 172.21.15.38:0/156977069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e40b3da0 msgr2=0x7f23e4069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.525+0000 7f23ec1ad700 1 --2- 172.21.15.38:0/156977069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e40b3da0 0x7f23e4069860 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f23e40682a0 tx=0x7f23d8017000 comp rx=0 tx=0).stop 2024-08-15T18:01:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.525+0000 7f23ea74a700 1 -- 172.21.15.38:0/156977069 reap_dead start 2024-08-15T18:01:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.525+0000 7f23ec1ad700 1 -- 172.21.15.38:0/156977069 shutdown_connections 2024-08-15T18:01:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.525+0000 7f23ec1ad700 1 -- 172.21.15.38:0/156977069 >> 172.21.15.38:0/156977069 conn(0x7f23e406ce20 msgr2=0x7f23e40adb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.525+0000 7f23ec1ad700 1 -- 172.21.15.38:0/156977069 shutdown_connections 2024-08-15T18:01:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.525+0000 7f23ec1ad700 1 -- 172.21.15.38:0/156977069 wait complete. 2024-08-15T18:01:58.540 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-08-15T18:01:58.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.649+0000 7f8018c32700 1 Processor -- start 2024-08-15T18:01:58.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.649+0000 7f8018c32700 1 -- start start 2024-08-15T18:01:58.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.649+0000 7f8018c32700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f80140af5f0 0x7f80140acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.649+0000 7f8018c32700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80140ad8a0 con 0x7f8014155d80 2024-08-15T18:01:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.649+0000 7f8018c32700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80140adb80 con 0x7f80140ad090 2024-08-15T18:01:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.649+0000 7f8018c32700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80140ade60 con 0x7f80140af5f0 2024-08-15T18:01:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.649+0000 7f80137fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f8014155d80 0x7f80140aeee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41344/0 (socket says 172.21.15.38:41344) 2024-08-15T18:01:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.649+0000 7f80137fe700 1 -- 172.21.15.38:0/4165736593 learned_addr learned my addr 172.21.15.38:0/4165736593 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.649+0000 7f8012ffd700 1 --2- 172.21.15.38:0/4165736593 >> v2:172.21.15.104:3300/0 conn(0x7f80140af5f0 0x7f80140acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.649+0000 7f80127fc700 1 -- 172.21.15.38:0/4165736593 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152897955 0 0) 0x7f80140ad8a0 con 0x7f8014155d80 2024-08-15T18:01:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f80127fc700 1 -- 172.21.15.38:0/4165736593 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ffc003570 con 0x7f8014155d80 2024-08-15T18:01:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f80127fc700 1 -- 172.21.15.38:0/4165736593 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278000386 0 0) 0x7f80140adb80 con 0x7f80140ad090 2024-08-15T18:01:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f80127fc700 1 -- 172.21.15.38:0/4165736593 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80140ad8a0 con 0x7f80140ad090 2024-08-15T18:01:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f8012ffd700 1 -- 172.21.15.38:0/4165736593 >> v1:172.21.15.104:6789/0 conn(0x7f80140ad090 legacy=0x7f8014157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f8012ffd700 1 -- 172.21.15.38:0/4165736593 >> v1:172.21.15.38:6789/0 conn(0x7f8014155d80 legacy=0x7f80140aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f8012ffd700 1 -- 172.21.15.38:0/4165736593 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140ae140 con 0x7f80140af5f0 2024-08-15T18:01:58.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f8012ffd700 1 --2- 172.21.15.38:0/4165736593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80140af5f0 0x7f80140acb50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f80080098e0 tx=0x7f80080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5b4a1ae74f6d61c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f80127fc700 1 -- 172.21.15.38:0/4165736593 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008011630 con 0x7f80140af5f0 2024-08-15T18:01:58.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f80127fc700 1 -- 172.21.15.38:0/4165736593 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8008011790 con 0x7f80140af5f0 2024-08-15T18:01:58.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f80127fc700 1 -- 172.21.15.38:0/4165736593 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008011b30 con 0x7f80140af5f0 2024-08-15T18:01:58.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f8018c32700 1 -- 172.21.15.38:0/4165736593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80140af5f0 msgr2=0x7f80140acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f8018c32700 1 --2- 172.21.15.38:0/4165736593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80140af5f0 0x7f80140acb50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f80080098e0 tx=0x7f80080091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f8018c32700 1 -- 172.21.15.38:0/4165736593 shutdown_connections 2024-08-15T18:01:58.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f8018c32700 1 --2- 172.21.15.38:0/4165736593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80140af5f0 0x7f80140acb50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f8018c32700 1 -- 172.21.15.38:0/4165736593 >> 172.21.15.38:0/4165736593 conn(0x7f801406e070 msgr2=0x7f80140b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:58.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f8018c32700 1 -- 172.21.15.38:0/4165736593 shutdown_connections 2024-08-15T18:01:58.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.653+0000 7f8018c32700 1 -- 172.21.15.38:0/4165736593 wait complete. 2024-08-15T18:01:58.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8018c32700 1 Processor -- start 2024-08-15T18:01:58.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8018c32700 1 -- start start 2024-08-15T18:01:58.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8018c32700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80140ad090 0x7f80141361c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8018c32700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f80140af5f0 0x7f8014151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8018c32700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8014155d80 0x7f80141518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8018c32700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80141492f0 con 0x7f8014155d80 2024-08-15T18:01:58.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8018c32700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8014149540 con 0x7f80140ad090 2024-08-15T18:01:58.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8018c32700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8014149820 con 0x7f80140af5f0 2024-08-15T18:01:58.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8013fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8014155d80 0x7f80141518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f80137fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80140ad090 0x7f80141361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f80137fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80140ad090 0x7f80141361c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.38:58966/0 (socket says 172.21.15.38:58966) 2024-08-15T18:01:58.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f80137fe700 1 -- 172.21.15.38:0/2671829891 learned_addr learned my addr 172.21.15.38:0/2671829891 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:58.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8013fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8014155d80 0x7f80141518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38068/0 (socket says 172.21.15.38:38068) 2024-08-15T18:01:58.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8012ffd700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f80140af5f0 0x7f8014151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8013fff700 1 -- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f80140af5f0 msgr2=0x7f8014151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8013fff700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f80140af5f0 0x7f8014151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8013fff700 1 -- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80140ad090 msgr2=0x7f80141361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8013fff700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80140ad090 0x7f80141361c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f8013fff700 1 -- 172.21.15.38:0/2671829891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8008009420 con 0x7f8014155d80 2024-08-15T18:01:58.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.657+0000 7f80137fe700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80140ad090 0x7f80141361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:01:58.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.661+0000 7f8013fff700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8014155d80 0x7f80141518d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c8e0 tx=0x7f800c0026f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.661+0000 7f7ffb7fe700 1 -- 172.21.15.38:0/2671829891 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c012630 con 0x7f8014155d80 2024-08-15T18:01:58.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.661+0000 7f7ffb7fe700 1 -- 172.21.15.38:0/2671829891 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c012790 con 0x7f8014155d80 2024-08-15T18:01:58.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.661+0000 7f8018c32700 1 -- 172.21.15.38:0/2671829891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8014149b00 con 0x7f8014155d80 2024-08-15T18:01:58.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.661+0000 7f7ffb7fe700 1 -- 172.21.15.38:0/2671829891 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c012b50 con 0x7f8014155d80 2024-08-15T18:01:58.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.661+0000 7f8018c32700 1 -- 172.21.15.38:0/2671829891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801414a0c0 con 0x7f8014155d80 2024-08-15T18:01:58.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.661+0000 7f8018c32700 1 -- 172.21.15.38:0/2671829891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80140638f0 con 0x7f8014155d80 2024-08-15T18:01:58.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.661+0000 7f7ffb7fe700 1 -- 172.21.15.38:0/2671829891 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f800c01a3b0 con 0x7f8014155d80 2024-08-15T18:01:58.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.665+0000 7f7ffb7fe700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8000060860 0x7f8000062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.665+0000 7f7ffb7fe700 1 -- 172.21.15.38:0/2671829891 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f800c07c9b0 con 0x7f8014155d80 2024-08-15T18:01:58.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.665+0000 7f80137fe700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8000060860 0x7f8000062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.665+0000 7f80137fe700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8000060860 0x7f8000062d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80040058d0 tx=0x7f8004007f10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.673+0000 7f7ffb7fe700 1 -- 172.21.15.38:0/2671829891 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f800c04d000 con 0x7f8014155d80 2024-08-15T18:01:58.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.841+0000 7f8018c32700 1 -- 172.21.15.38:0/2671829891 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f80140af3d0 con 0x7f8000060860 2024-08-15T18:01:58.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.841+0000 7f7ffb7fe700 1 -- 172.21.15.38:0/2671829891 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f80140af3d0 con 0x7f8000060860 2024-08-15T18:01:58.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 -- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8000060860 msgr2=0x7f8000062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8000060860 0x7f8000062d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80040058d0 tx=0x7f8004007f10 comp rx=0 tx=0).stop 2024-08-15T18:01:58.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 -- 172.21.15.38:0/2671829891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8014155d80 msgr2=0x7f80141518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8014155d80 0x7f80141518d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c8e0 tx=0x7f800c0026f0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 -- 172.21.15.38:0/2671829891 shutdown_connections 2024-08-15T18:01:58.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f8000060860 0x7f8000062d10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8014155d80 0x7f80141518d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f80140af5f0 0x7f8014151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 --2- 172.21.15.38:0/2671829891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80140ad090 0x7f80141361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 -- 172.21.15.38:0/2671829891 >> 172.21.15.38:0/2671829891 conn(0x7f801406e070 msgr2=0x7f8014148740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:58.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 -- 172.21.15.38:0/2671829891 shutdown_connections 2024-08-15T18:01:58.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.845+0000 7f8018c32700 1 -- 172.21.15.38:0/2671829891 wait complete. 2024-08-15T18:01:58.850 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 6 to deep-scrub 2024-08-15T18:01:58.860 INFO:tasks.ceph:Scrubbing osd.7 2024-08-15T18:01:58.860 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-08-15T18:01:58.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.973+0000 7fdffcb25700 1 Processor -- start 2024-08-15T18:01:58.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.973+0000 7fdffcb25700 1 -- start start 2024-08-15T18:01:58.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdff80c6160 0x7fdff80caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdff80cb3f0 0x7fdff80ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80cc480 con 0x7fdff80cd2b0 2024-08-15T18:01:58.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff8159720 con 0x7fdff80c6160 2024-08-15T18:01:58.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80cb870 con 0x7fdff80cb3f0 2024-08-15T18:01:58.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff5d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdff80c6160 0x7fdff80caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff659c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdff80cd2b0 0x7fdff80cd680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41346/0 (socket says 172.21.15.38:41346) 2024-08-15T18:01:58.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff659c700 1 -- 172.21.15.38:0/3355007005 learned_addr learned my addr 172.21.15.38:0/3355007005 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:58.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff6d9d700 1 --2- 172.21.15.38:0/3355007005 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdff80cb3f0 0x7fdff80ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff5d9b700 1 -- 172.21.15.38:0/3355007005 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdff80cb3f0 msgr2=0x7fdff80ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:58.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff5d9b700 1 --2- 172.21.15.38:0/3355007005 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdff80cb3f0 0x7fdff80ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff5d9b700 1 -- 172.21.15.38:0/3355007005 >> v1:172.21.15.38:6789/0 conn(0x7fdff80cd2b0 legacy=0x7fdff80cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff5d9b700 1 -- 172.21.15.38:0/3355007005 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff80cbb50 con 0x7fdff80c6160 2024-08-15T18:01:58.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff5d9b700 1 --2- 172.21.15.38:0/3355007005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdff80c6160 0x7fdff80caeb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00098e0 tx=0x7fdfe00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95820180be849d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff559a700 1 -- 172.21.15.38:0/3355007005 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000b640 con 0x7fdff80c6160 2024-08-15T18:01:58.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff559a700 1 -- 172.21.15.38:0/3355007005 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe000b7a0 con 0x7fdff80c6160 2024-08-15T18:01:58.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdff559a700 1 -- 172.21.15.38:0/3355007005 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000bb40 con 0x7fdff80c6160 2024-08-15T18:01:58.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 -- 172.21.15.38:0/3355007005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdff80c6160 msgr2=0x7fdff80caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 --2- 172.21.15.38:0/3355007005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdff80c6160 0x7fdff80caeb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00098e0 tx=0x7fdfe00091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 -- 172.21.15.38:0/3355007005 shutdown_connections 2024-08-15T18:01:58.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 --2- 172.21.15.38:0/3355007005 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdff80cb3f0 0x7fdff80ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 --2- 172.21.15.38:0/3355007005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdff80c6160 0x7fdff80caeb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 -- 172.21.15.38:0/3355007005 >> 172.21.15.38:0/3355007005 conn(0x7fdff8099550 msgr2=0x7fdff8067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:58.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 -- 172.21.15.38:0/3355007005 shutdown_connections 2024-08-15T18:01:58.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.977+0000 7fdffcb25700 1 -- 172.21.15.38:0/3355007005 wait complete. 2024-08-15T18:01:58.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdffcb25700 1 Processor -- start 2024-08-15T18:01:58.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdffcb25700 1 -- start start 2024-08-15T18:01:58.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdffcb25700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdff80c6160 0x7fdff80b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdffcb25700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdff80cd2b0 0x7fdff81569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdffcb25700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdff80ae1e0 0x7fdff80ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdffcb25700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80b5020 con 0x7fdff80cd2b0 2024-08-15T18:01:58.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdffcb25700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80aebc0 con 0x7fdff80c6160 2024-08-15T18:01:58.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdffcb25700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80b3700 con 0x7fdff80ae1e0 2024-08-15T18:01:58.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdff5d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdff80cd2b0 0x7fdff81569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdff5d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdff80cd2b0 0x7fdff81569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38090/0 (socket says 172.21.15.38:38090) 2024-08-15T18:01:58.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdff5d9b700 1 -- 172.21.15.38:0/3581916636 learned_addr learned my addr 172.21.15.38:0/3581916636 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:58.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdff659c700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdff80c6160 0x7fdff80b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.981+0000 7fdff6d9d700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdff80ae1e0 0x7fdff80ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdff5d9b700 1 -- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdff80ae1e0 msgr2=0x7fdff80ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdff5d9b700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdff80ae1e0 0x7fdff80ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdff5d9b700 1 -- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdff80c6160 msgr2=0x7fdff80b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:58.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdff5d9b700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdff80c6160 0x7fdff80b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:58.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdff5d9b700 1 -- 172.21.15.38:0/3581916636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe0009420 con 0x7fdff80cd2b0 2024-08-15T18:01:58.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdff5d9b700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdff80cd2b0 0x7fdff81569b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000b6c0 tx=0x7fdfe0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdfee7fc700 1 -- 172.21.15.38:0/3581916636 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000bb20 con 0x7fdff80cd2b0 2024-08-15T18:01:58.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdfee7fc700 1 -- 172.21.15.38:0/3581916636 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe0000c90 con 0x7fdff80cd2b0 2024-08-15T18:01:58.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdffcb25700 1 -- 172.21.15.38:0/3581916636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff80b30f0 con 0x7fdff80cd2b0 2024-08-15T18:01:58.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdfee7fc700 1 -- 172.21.15.38:0/3581916636 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0011650 con 0x7fdff80cd2b0 2024-08-15T18:01:58.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdffcb25700 1 -- 172.21.15.38:0/3581916636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff80b3390 con 0x7fdff80cd2b0 2024-08-15T18:01:58.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdffcb25700 1 -- 172.21.15.38:0/3581916636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdff80638f0 con 0x7fdff80cd2b0 2024-08-15T18:01:58.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.985+0000 7fdfee7fc700 1 -- 172.21.15.38:0/3581916636 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdfe0011a80 con 0x7fdff80cd2b0 2024-08-15T18:01:58.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.989+0000 7fdfee7fc700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdfd8060860 0x7fdfd8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.989+0000 7fdfee7fc700 1 -- 172.21.15.38:0/3581916636 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fdfe007e270 con 0x7fdff80cd2b0 2024-08-15T18:01:58.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.989+0000 7fdfee7fc700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] conn(0x7fdfd8066750 0x7fdfd8068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:58.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.989+0000 7fdfee7fc700 1 -- 172.21.15.38:0/3581916636 --> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfd8069210 con 0x7fdfd8066750 2024-08-15T18:01:58.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.989+0000 7fdfee7fc700 1 -- 172.21.15.38:0/3581916636 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfe00808b0 con 0x7fdff80cd2b0 2024-08-15T18:01:58.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.989+0000 7fdff659c700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdfd8060860 0x7fdfd8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.989+0000 7fdff6d9d700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] conn(0x7fdfd8066750 0x7fdfd8068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:58.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.989+0000 7fdff659c700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdfd8060860 0x7fdfd8062d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8002250 tx=0x7fdfe8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.989+0000 7fdff6d9d700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] conn(0x7fdfd8066750 0x7fdfd8068b60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:58.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:58.989+0000 7fdfee7fc700 1 -- 172.21.15.38:0/3581916636 <== osd.7 v2:172.21.15.104:6816/24504 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fdfd8069210 con 0x7fdfd8066750 2024-08-15T18:01:59.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.009+0000 7fdffcb25700 1 -- 172.21.15.38:0/3581916636 --> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdff8068020 con 0x7fdfd8066750 2024-08-15T18:01:59.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdfee7fc700 1 -- 172.21.15.38:0/3581916636 <== osd.7 v2:172.21.15.104:6816/24504 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdff8068020 con 0x7fdfd8066750 2024-08-15T18:01:59.015 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-08-15T18:01:59.016 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T18:01:59.016 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-08-15T18:01:59.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdffcb25700 1 -- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] conn(0x7fdfd8066750 msgr2=0x7fdfd8068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdffcb25700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:6816/24504,v1:172.21.15.104:6817/24504] conn(0x7fdfd8066750 0x7fdfd8068b60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdffcb25700 1 -- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdfd8060860 msgr2=0x7fdfd8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdffcb25700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7fdfd8060860 0x7fdfd8062d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8002250 tx=0x7fdfe8009040 comp rx=0 tx=0).stop 2024-08-15T18:01:59.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdffcb25700 1 -- 172.21.15.38:0/3581916636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdff80cd2b0 msgr2=0x7fdff81569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdffcb25700 1 --2- 172.21.15.38:0/3581916636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdff80cd2b0 0x7fdff81569b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000b6c0 tx=0x7fdfe0017000 comp rx=0 tx=0).stop 2024-08-15T18:01:59.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdff6d9d700 1 -- 172.21.15.38:0/3581916636 reap_dead start 2024-08-15T18:01:59.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdffcb25700 1 -- 172.21.15.38:0/3581916636 shutdown_connections 2024-08-15T18:01:59.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdffcb25700 1 -- 172.21.15.38:0/3581916636 >> 172.21.15.38:0/3581916636 conn(0x7fdff8099550 msgr2=0x7fdff80ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:59.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdffcb25700 1 -- 172.21.15.38:0/3581916636 shutdown_connections 2024-08-15T18:01:59.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.013+0000 7fdffcb25700 1 -- 172.21.15.38:0/3581916636 wait complete. 2024-08-15T18:01:59.029 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-08-15T18:01:59.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.137+0000 7f82a6e2a700 1 Processor -- start 2024-08-15T18:01:59.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.137+0000 7f82a6e2a700 1 -- start start 2024-08-15T18:01:59.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.137+0000 7f82a6e2a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82a00ae2a0 0x7f82a00ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.137+0000 7f82a6e2a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f82a00b0f10 0x7f82a00a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.137+0000 7f82a6e2a700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a00a9160 con 0x7f82a015a620 2024-08-15T18:01:59.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.137+0000 7f82a6e2a700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a00aff10 con 0x7f82a00b0f10 2024-08-15T18:01:59.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.137+0000 7f82a6e2a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00b1680 con 0x7f82a00ae2a0 2024-08-15T18:01:59.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a4bc6700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f82a015a620 0x7f82a00adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41358/0 (socket says 172.21.15.38:41358) 2024-08-15T18:01:59.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a4bc6700 1 -- 172.21.15.38:0/1510788805 learned_addr learned my addr 172.21.15.38:0/1510788805 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:01:59.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a53c7700 1 --2- 172.21.15.38:0/1510788805 >> v2:172.21.15.104:3300/0 conn(0x7f82a00b0f10 0x7f82a00a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f829ffff700 1 --2- 172.21.15.38:0/1510788805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82a00ae2a0 0x7f82a00ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f829f7fe700 1 -- 172.21.15.38:0/1510788805 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782123195 0 0) 0x7f82a00a9160 con 0x7f82a015a620 2024-08-15T18:01:59.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f829f7fe700 1 -- 172.21.15.38:0/1510788805 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8288003570 con 0x7f82a015a620 2024-08-15T18:01:59.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a53c7700 1 -- 172.21.15.38:0/1510788805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82a00ae2a0 msgr2=0x7f82a00ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:59.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a53c7700 1 --2- 172.21.15.38:0/1510788805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82a00ae2a0 0x7f82a00ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a53c7700 1 -- 172.21.15.38:0/1510788805 >> v1:172.21.15.38:6789/0 conn(0x7f82a015a620 legacy=0x7f82a00adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a53c7700 1 -- 172.21.15.38:0/1510788805 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a00b1960 con 0x7f82a00b0f10 2024-08-15T18:01:59.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a53c7700 1 --2- 172.21.15.38:0/1510788805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82a00b0f10 0x7f82a00a95f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82900098e0 tx=0x7f82900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d29e838bffd5fe server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:59.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f829f7fe700 1 -- 172.21.15.38:0/1510788805 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290011630 con 0x7f82a00b0f10 2024-08-15T18:01:59.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f829f7fe700 1 -- 172.21.15.38:0/1510788805 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8290011790 con 0x7f82a00b0f10 2024-08-15T18:01:59.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f829f7fe700 1 -- 172.21.15.38:0/1510788805 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290011b30 con 0x7f82a00b0f10 2024-08-15T18:01:59.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1510788805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82a00b0f10 msgr2=0x7f82a00a95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a6e2a700 1 --2- 172.21.15.38:0/1510788805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82a00b0f10 0x7f82a00a95f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82900098e0 tx=0x7f82900091a0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1510788805 shutdown_connections 2024-08-15T18:01:59.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a6e2a700 1 --2- 172.21.15.38:0/1510788805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82a00b0f10 0x7f82a00a95f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a6e2a700 1 --2- 172.21.15.38:0/1510788805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82a00ae2a0 0x7f82a00ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1510788805 >> 172.21.15.38:0/1510788805 conn(0x7f82a0067810 msgr2=0x7f82a0067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:59.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1510788805 shutdown_connections 2024-08-15T18:01:59.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1510788805 wait complete. 2024-08-15T18:01:59.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.141+0000 7f82a6e2a700 1 Processor -- start 2024-08-15T18:01:59.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a6e2a700 1 -- start start 2024-08-15T18:01:59.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a6e2a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82a00ae2a0 0x7f82a0156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a6e2a700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82a00b0f10 0x7f82a015d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a6e2a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82a015a620 0x7f82a0098b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a6e2a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0167020 con 0x7f82a00ae2a0 2024-08-15T18:01:59.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a6e2a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0156740 con 0x7f82a015a620 2024-08-15T18:01:59.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a6e2a700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0167210 con 0x7f82a00b0f10 2024-08-15T18:01:59.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a4bc6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82a00ae2a0 0x7f82a0156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a4bc6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82a00ae2a0 0x7f82a0156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38096/0 (socket says 172.21.15.38:38096) 2024-08-15T18:01:59.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a4bc6700 1 -- 172.21.15.38:0/1082027380 learned_addr learned my addr 172.21.15.38:0/1082027380 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:59.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a53c7700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82a015a620 0x7f82a0098b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f829ffff700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82a00b0f10 0x7f82a015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a4bc6700 1 -- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82a00b0f10 msgr2=0x7f82a015d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a4bc6700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82a00b0f10 0x7f82a015d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a4bc6700 1 -- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82a015a620 msgr2=0x7f82a0098b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a4bc6700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82a015a620 0x7f82a0098b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a4bc6700 1 -- 172.21.15.38:0/1082027380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290009420 con 0x7f82a00ae2a0 2024-08-15T18:01:59.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a4bc6700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82a00ae2a0 0x7f82a0156200 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f829400c590 tx=0x7f8294007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:59.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f829cff9700 1 -- 172.21.15.38:0/1082027380 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294005b30 con 0x7f82a00ae2a0 2024-08-15T18:01:59.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f829cff9700 1 -- 172.21.15.38:0/1082027380 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8294005cb0 con 0x7f82a00ae2a0 2024-08-15T18:01:59.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1082027380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a01674b0 con 0x7f82a00ae2a0 2024-08-15T18:01:59.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f829cff9700 1 -- 172.21.15.38:0/1082027380 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294007320 con 0x7f82a00ae2a0 2024-08-15T18:01:59.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.145+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1082027380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a0167d50 con 0x7f82a00ae2a0 2024-08-15T18:01:59.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.149+0000 7f829cff9700 1 -- 172.21.15.38:0/1082027380 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f829401b3b0 con 0x7f82a00ae2a0 2024-08-15T18:01:59.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.149+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1082027380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a00645c0 con 0x7f82a00ae2a0 2024-08-15T18:01:59.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.149+0000 7f829cff9700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f828c060860 0x7f828c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.149+0000 7f829cff9700 1 -- 172.21.15.38:0/1082027380 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f829407bf50 con 0x7f82a00ae2a0 2024-08-15T18:01:59.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.149+0000 7f829ffff700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f828c060860 0x7f828c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.149+0000 7f829ffff700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f828c060860 0x7f828c062d10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82a00997e0 tx=0x7f829800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:59.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.157+0000 7f829cff9700 1 -- 172.21.15.38:0/1082027380 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f829404c640 con 0x7f82a00ae2a0 2024-08-15T18:01:59.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.321+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1082027380 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f82a0157880 con 0x7f828c060860 2024-08-15T18:01:59.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.321+0000 7f829cff9700 1 -- 172.21.15.38:0/1082027380 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f82a0157880 con 0x7f828c060860 2024-08-15T18:01:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f828c060860 msgr2=0x7f828c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f828c060860 0x7f828c062d10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82a00997e0 tx=0x7f829800b040 comp rx=0 tx=0).stop 2024-08-15T18:01:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1082027380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82a00ae2a0 msgr2=0x7f82a0156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82a00ae2a0 0x7f82a0156200 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f829400c590 tx=0x7f8294007e00 comp rx=0 tx=0).stop 2024-08-15T18:01:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1082027380 shutdown_connections 2024-08-15T18:01:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f828c060860 0x7f828c062d10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82a015a620 0x7f82a0098b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82a00b0f10 0x7f82a015d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 --2- 172.21.15.38:0/1082027380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82a00ae2a0 0x7f82a0156200 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1082027380 >> 172.21.15.38:0/1082027380 conn(0x7f82a0067810 msgr2=0x7f82a0157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:59.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1082027380 shutdown_connections 2024-08-15T18:01:59.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.325+0000 7f82a6e2a700 1 -- 172.21.15.38:0/1082027380 wait complete. 2024-08-15T18:01:59.332 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 7 to deep-scrub 2024-08-15T18:01:59.346 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T18:01:59.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2bfff700 1 Processor -- start 2024-08-15T18:01:59.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2bfff700 1 -- start start 2024-08-15T18:01:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6f2c14f930 0x7f6f2c0b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2bfff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6f2c068db0 0x7f6f2c0ad500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2bfff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2c155b30 con 0x7f6f2c0b21b0 2024-08-15T18:01:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2bfff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2c0af730 con 0x7f6f2c14f930 2024-08-15T18:01:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2bfff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2c155df0 con 0x7f6f2c068db0 2024-08-15T18:01:59.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2affd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6f2c14f930 0x7f6f2c0b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2affd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6f2c14f930 0x7f6f2c0b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38106/0 (socket says 172.21.15.38:38106) 2024-08-15T18:01:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2affd700 1 -- 172.21.15.38:0/431088923 learned_addr learned my addr 172.21.15.38:0/431088923 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2b7fe700 1 --2- 172.21.15.38:0/431088923 >> v2:172.21.15.104:3300/0 conn(0x7f6f2c068db0 0x7f6f2c0ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2affd700 1 -- 172.21.15.38:0/431088923 >> v2:172.21.15.104:3300/0 conn(0x7f6f2c068db0 msgr2=0x7f6f2c0ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:01:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2affd700 1 --2- 172.21.15.38:0/431088923 >> v2:172.21.15.104:3300/0 conn(0x7f6f2c068db0 0x7f6f2c0ad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2affd700 1 -- 172.21.15.38:0/431088923 >> v1:172.21.15.104:6789/0 conn(0x7f6f2c0b21b0 legacy=0x7f6f2c0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f29ffb700 1 -- 172.21.15.38:0/431088923 <== mon.1 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346212279 0 0) 0x7f6f2c155b30 con 0x7f6f2c0b21b0 2024-08-15T18:01:59.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2affd700 1 -- 172.21.15.38:0/431088923 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c1560d0 con 0x7f6f2c14f930 2024-08-15T18:01:59.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f2affd700 1 --2- 172.21.15.38:0/431088923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f2c14f930 0x7f6f2c0b1c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c153700 tx=0x7f6f1400a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9f34aff3fba8dc server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:59.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f29ffb700 1 -- 172.21.15.38:0/431088923 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400ca40 con 0x7f6f2c14f930 2024-08-15T18:01:59.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f29ffb700 1 -- 172.21.15.38:0/431088923 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1400cba0 con 0x7f6f2c14f930 2024-08-15T18:01:59.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.449+0000 7f6f29ffb700 1 -- 172.21.15.38:0/431088923 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14018400 con 0x7f6f2c14f930 2024-08-15T18:01:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.453+0000 7f6f2bfff700 1 -- 172.21.15.38:0/431088923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f2c14f930 msgr2=0x7f6f2c0b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.453+0000 7f6f2bfff700 1 --2- 172.21.15.38:0/431088923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f2c14f930 0x7f6f2c0b1c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c153700 tx=0x7f6f1400a3f0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.453+0000 7f6f2bfff700 1 -- 172.21.15.38:0/431088923 shutdown_connections 2024-08-15T18:01:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.453+0000 7f6f2bfff700 1 --2- 172.21.15.38:0/431088923 >> v2:172.21.15.104:3300/0 conn(0x7f6f2c068db0 0x7f6f2c0ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.453+0000 7f6f2bfff700 1 --2- 172.21.15.38:0/431088923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f2c14f930 0x7f6f2c0b1c70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.453+0000 7f6f2bfff700 1 -- 172.21.15.38:0/431088923 >> 172.21.15.38:0/431088923 conn(0x7f6f2c06d790 msgr2=0x7f6f2c06db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:59.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.453+0000 7f6f2bfff700 1 -- 172.21.15.38:0/431088923 shutdown_connections 2024-08-15T18:01:59.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.453+0000 7f6f2bfff700 1 -- 172.21.15.38:0/431088923 wait complete. 2024-08-15T18:01:59.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.453+0000 7f6f2bfff700 1 Processor -- start 2024-08-15T18:01:59.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.453+0000 7f6f2bfff700 1 -- start start 2024-08-15T18:01:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2bfff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f2c068db0 0x7f6f2c13d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f2c0b21b0 0x7f6f2c14e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f2c14f930 0x7f6f2c14e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2bfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1525f0 con 0x7f6f2c14f930 2024-08-15T18:01:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2bfff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c13d570 con 0x7f6f2c0b21b0 2024-08-15T18:01:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2bfff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1519e0 con 0x7f6f2c068db0 2024-08-15T18:01:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f2c14f930 0x7f6f2c14e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f2c14f930 0x7f6f2c14e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38120/0 (socket says 172.21.15.38:38120) 2024-08-15T18:01:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2b7fe700 1 -- 172.21.15.38:0/4448831 learned_addr learned my addr 172.21.15.38:0/4448831 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:01:59.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2affd700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f2c068db0 0x7f6f2c13d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2a7fc700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f2c0b21b0 0x7f6f2c14e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2b7fe700 1 -- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f2c068db0 msgr2=0x7f6f2c13d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2b7fe700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f2c068db0 0x7f6f2c13d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2b7fe700 1 -- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f2c0b21b0 msgr2=0x7f6f2c14e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2b7fe700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f2c0b21b0 0x7f6f2c14e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2b7fe700 1 -- 172.21.15.38:0/4448831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1400a690 con 0x7f6f2c14f930 2024-08-15T18:01:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2b7fe700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f2c14f930 0x7f6f2c14e660 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400fbf0 tx=0x7f6f24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f22ffd700 1 -- 172.21.15.38:0/4448831 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f240078f0 con 0x7f6f2c14f930 2024-08-15T18:01:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f22ffd700 1 -- 172.21.15.38:0/4448831 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f24007a50 con 0x7f6f2c14f930 2024-08-15T18:01:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2bfff700 1 -- 172.21.15.38:0/4448831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c151cc0 con 0x7f6f2c14f930 2024-08-15T18:01:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f22ffd700 1 -- 172.21.15.38:0/4448831 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24007e10 con 0x7f6f2c14f930 2024-08-15T18:01:59.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.457+0000 7f6f2bfff700 1 -- 172.21.15.38:0/4448831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c152280 con 0x7f6f2c14f930 2024-08-15T18:01:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.461+0000 7f6f2bfff700 1 -- 172.21.15.38:0/4448831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2c1663d0 con 0x7f6f2c14f930 2024-08-15T18:01:59.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.461+0000 7f6f22ffd700 1 -- 172.21.15.38:0/4448831 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6f24011660 con 0x7f6f2c14f930 2024-08-15T18:01:59.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.461+0000 7f6f22ffd700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f10060860 0x7f6f10062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:01:59.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.461+0000 7f6f22ffd700 1 -- 172.21.15.38:0/4448831 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f6f2407ef20 con 0x7f6f2c14f930 2024-08-15T18:01:59.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.461+0000 7f6f2affd700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f10060860 0x7f6f10062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:01:59.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.461+0000 7f6f2affd700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f10060860 0x7f6f10062d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c153700 tx=0x7f6f14018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:01:59.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.465+0000 7f6f22ffd700 1 -- 172.21.15.38:0/4448831 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6f24053160 con 0x7f6f2c14f930 2024-08-15T18:01:59.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.625+0000 7f6f2bfff700 1 -- 172.21.15.38:0/4448831 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f2c0643a0 con 0x7f6f10060860 2024-08-15T18:01:59.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f22ffd700 1 -- 172.21.15.38:0/4448831 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48767 (secure 0 0 0) 0x7f6f2c0643a0 con 0x7f6f10060860 2024-08-15T18:01:59.636 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:01:59.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 -- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f10060860 msgr2=0x7f6f10062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f10060860 0x7f6f10062d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c153700 tx=0x7f6f14018000 comp rx=0 tx=0).stop 2024-08-15T18:01:59.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 -- 172.21.15.38:0/4448831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f2c14f930 msgr2=0x7f6f2c14e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:01:59.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f2c14f930 0x7f6f2c14e660 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400fbf0 tx=0x7f6f24008040 comp rx=0 tx=0).stop 2024-08-15T18:01:59.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 -- 172.21.15.38:0/4448831 shutdown_connections 2024-08-15T18:01:59.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6f10060860 0x7f6f10062d10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f2c14f930 0x7f6f2c14e660 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f2c0b21b0 0x7f6f2c14e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 --2- 172.21.15.38:0/4448831 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f2c068db0 0x7f6f2c13d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:01:59.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 -- 172.21.15.38:0/4448831 >> 172.21.15.38:0/4448831 conn(0x7f6f2c06d790 msgr2=0x7f6f2c156ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:01:59.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 -- 172.21.15.38:0/4448831 shutdown_connections 2024-08-15T18:01:59.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:01:59.633+0000 7f6f2bfff700 1 -- 172.21.15.38:0/4448831 wait complete. 2024-08-15T18:01:59.640 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-08-15T18:01:59.651 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-08-15T18:01:58.247615+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2408920,"kb_used_data":5096,"kb_used_omap":13,"kb_used_meta":2403698,"kb_avail":752565800,"statfs":{"total":773094113280,"available":770627379200,"internally_reserved":0,"allocated":5218304,"data_stored":2860392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2461387456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.133489"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-08-15T18:01:50.118034+0000","last_change":"2024-08-15T17:58:24.394014+0000","last_active":"2024-08-15T18:01:50.118034+0000","last_peered":"2024-08-15T18:01:50.118034+0000","last_clean":"2024-08-15T18:01:50.118034+0000","last_became_active":"2024-08-15T17:58:24.391735+0000","last_became_peered":"2024-08-15T17:58:24.391735+0000","last_unstale":"2024-08-15T18:01:50.118034+0000","last_undegraded":"2024-08-15T18:01:50.118034+0000","last_fullsized":"2024-08-15T18:01:50.118034+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:58:22.996335+0000","last_clean_scrub_stamp":"2024-08-15T17:58:22.996335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:31.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302592,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069248,"statfs":{"total":96636764160,"available":96326909952,"internally_reserved":0,"allocated":1097728,"data_stored":800325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92600000000000005,"15min":0.92800000000000005},"min":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1919999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.107},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.86299999999999999,"15min":0.871},"min":{"1min":0.64300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.208,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.748}]},{"osd":1,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91800000000000004,"15min":0.90700000000000003},"min":{"1min":0.67300000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89200000000000002,"15min":0.875},"min":{"1min":0.76900000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0880000000000001,"5min":1.165,"15min":1.165},"last":0.96799999999999997}]},{"osd":2,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.73099999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.262,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.258},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1,"15min":1.022},"min":{"1min":0.65400000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.111,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.0169999999999999}]},{"osd":3,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.94599999999999995,"15min":0.93999999999999995},"min":{"1min":0.73799999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0880000000000001,"15min":1.083},"min":{"1min":0.86199999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3160000000000001,"5min":1.415,"15min":1.415},"last":1.1870000000000001}]},{"osd":4,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.93999999999999995,"15min":0.96199999999999997},"min":{"1min":0.622,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.21,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.0529999999999999,"15min":1.0780000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2350000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.88900000000000001}]},{"osd":5,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.105,"15min":1.1319999999999999},"min":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3080000000000001,"5min":1.599,"15min":1.599},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.131,"15min":1.1519999999999999},"min":{"1min":0.78400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3660000000000001,"5min":1.486,"15min":1.486},"last":1.0820000000000001}]},{"osd":6,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1970000000000001,"15min":1.2170000000000001},"min":{"1min":0.85299999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.379,"5min":1.631,"15min":1.631},"last":1.216},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1850000000000001,"15min":1.1950000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.49,"5min":1.518,"15min":1.518},"last":1.3180000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85199999999999998,"15min":0.81999999999999995},"min":{"1min":0.58999999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.224,"5min":1.226,"15min":1.226},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.91400000000000003,"15min":0.93600000000000005},"min":{"1min":0.622,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1970000000000001,"5min":1.516,"15min":1.516},"last":0.95899999999999996}]},{"osd":1,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88600000000000001,"15min":0.90800000000000003},"min":{"1min":0.66000000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.054,"5min":1.333,"15min":1.333},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.99299999999999999,"15min":1.006},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2669999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.81299999999999994}]},{"osd":2,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.98299999999999998,"15min":1.0009999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.274,"5min":1.369,"15min":1.369},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.998,"5min":1.0109999999999999,"15min":1.02},"min":{"1min":0.80600000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2609999999999999,"5min":1.409,"15min":1.409},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.67500000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1419999999999999,"5min":1.302,"15min":1.302},"last":1.04},{"interface":"front","average":{"1min":1.032,"5min":1.022,"15min":1.026},"min":{"1min":0.83599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.26,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.95699999999999996,"15min":0.96599999999999997},"min":{"1min":0.63400000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.147,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.002},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.105,"15min":1.131},"min":{"1min":0.73299999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.306,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.876,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.234,"5min":1.345,"15min":1.345},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.161},"min":{"1min":0.999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.3240000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.88100000000000001}]},{"osd":7,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1970000000000001,"15min":1.232},"min":{"1min":0.97399999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.377,"5min":1.631,"15min":1.631},"last":1.119},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.159,"15min":1.171},"min":{"1min":1,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4770000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0680000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2310000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.745},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.95199999999999996,"15min":0.96699999999999997},"min":{"1min":0.65300000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2749999999999999,"5min":1.371,"15min":1.371},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.96699999999999997,"15min":0.98299999999999998},"min":{"1min":0.64300000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1060000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.194},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.98699999999999999,"15min":1.0309999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.167,"5min":1.708,"15min":1.708},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.998,"15min":1.03},"min":{"1min":0.72799999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2629999999999999,"5min":1.514,"15min":1.514},"last":0.873},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0429999999999999,"15min":1.0669999999999999},"min":{"1min":0.73099999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.2090000000000001,"5min":1.476,"15min":1.476},"last":0.78600000000000003}]},{"osd":3,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.1020000000000001,"15min":1.1399999999999999},"min":{"1min":0.82099999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.335,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.016},{"interface":"front","average":{"1min":0.92500000000000004,"5min":1.091,"15min":1.135},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3149999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.012,"15min":1.054},"min":{"1min":0.68500000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.208,"5min":1.419,"15min":1.419},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.137,"15min":1.1990000000000001},"min":{"1min":0.69699999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.3120000000000001,"5min":1.621,"15min":1.621},"last":1.222}]},{"osd":6,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.121,"15min":1.167},"min":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3240000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.198,"15min":1.264},"min":{"1min":0.82899999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.2629999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2789999999999999,"15min":1.353},"min":{"1min":0.86799999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3799999999999999,"5min":1.772,"15min":1.772},"last":1.115},{"interface":"front","average":{"1min":1.171,"5min":1.2989999999999999,"15min":1.345},"min":{"1min":0.91700000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.391,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.2509999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.71099999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.248,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.04,"15min":1.042},"min":{"1min":0.75600000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2470000000000001,"5min":1.383,"15min":1.383},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.66700000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96499999999999997,"15min":0.95699999999999996},"min":{"1min":0.71699999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2070000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.89300000000000002}]},{"osd":2,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95399999999999996,"15min":0.96099999999999997},"min":{"1min":0.76700000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.177,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.996,"15min":1.012},"min":{"1min":0.69699999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.2949999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.77500000000000002}]},{"osd":3,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.92400000000000004,"15min":0.92500000000000004},"min":{"1min":0.73399999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.026},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.01,"15min":0.98799999999999999},"min":{"1min":0.80900000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.3009999999999999}]},{"osd":4,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.206},{"interface":"front","average":{"1min":1.036,"5min":1.0189999999999999,"15min":1.0249999999999999},"min":{"1min":0.73099999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.054}]},{"osd":6,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.99199999999999999,"15min":0.99399999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1950000000000001,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.117,"15min":1.1080000000000001},"min":{"1min":0.997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.3460000000000001}]},{"osd":7,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.071,"15min":1.075},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.1679999999999999,"15min":1.165},"min":{"1min":0.997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.3420000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.1140000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.998,"15min":0.998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.99299999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.53000000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.03,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.69599999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.872,"15min":0.88200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.214,"5min":1.74,"15min":1.74},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.97899999999999998,"15min":0.99299999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.089,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.95299999999999996}]},{"osd":4,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1100000000000001,"15min":1.1319999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.444,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.022,"15min":1.0309999999999999},"min":{"1min":0.83599999999999997,"5min":0.755,"15min":0.755},"max":{"1min":1.4239999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.3979999999999999}]},{"osd":5,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.167,"15min":1.181},"min":{"1min":0.76700000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5649999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.131},{"interface":"front","average":{"1min":1.028,"5min":1.1220000000000001,"15min":1.1699999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.448,"5min":1.581,"15min":1.581},"last":1.2989999999999999}]},{"osd":6,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1819999999999999,"15min":1.204},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5529999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.099},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6160000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.232}]},{"osd":7,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2030000000000001,"15min":1.2250000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5589999999999999,"5min":2.073,"15min":2.073},"last":1.454},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2250000000000001,"15min":1.244},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.4890000000000001,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1819999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302016,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069824,"statfs":{"total":96636764160,"available":96327499776,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.105,"5min":1.204,"15min":1.204},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.61099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0589999999999999,"5min":1.111,"15min":1.111},"last":0.85599999999999998}]},{"osd":1,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.626,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1140000000000001,"5min":1.127,"15min":1.127},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2270000000000001,"5min":1.278,"15min":1.278},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.57899999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0009999999999999,"5min":1.073,"15min":1.073},"last":0.745},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.93000000000000005},"min":{"1min":0.73699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2090000000000001,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.71399999999999997}]},{"osd":4,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.0089999999999999,"15min":0.996},"min":{"1min":0.77500000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.296,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.085,"5min":1.083,"15min":1.0900000000000001},"min":{"1min":0.82599999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.335,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.121}]},{"osd":5,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.1359999999999999,"15min":1.155},"min":{"1min":0.85499999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.292,"5min":1.573,"15min":1.573},"last":1.018},{"interface":"front","average":{"1min":1.091,"5min":1.099,"15min":1.107},"min":{"1min":0.87,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3300000000000001,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.073}]},{"osd":6,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.137,"15min":1.159},"min":{"1min":0.81399999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.3480000000000001,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.1739999999999999,"15min":1.1830000000000001},"min":{"1min":0.84499999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.399,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.048}]},{"osd":7,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2110000000000001,"15min":1.218},"min":{"1min":0.96499999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.2290000000000001,"15min":1.244},"min":{"1min":1.0129999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4259999999999999,"5min":1.861,"15min":1.861},"last":0.91800000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.52300000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.443,"15min":0.443},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90600000000000003,"15min":0.89100000000000001},"min":{"1min":0.70699999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90100000000000002,"15min":0.92400000000000004},"min":{"1min":0.66300000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0700000000000001,"5min":1.778,"15min":1.778},"last":0.81999999999999995}]},{"osd":4,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.069,"15min":1.093},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4199999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0529999999999999,"15min":1.056},"min":{"1min":0.79200000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.4750000000000001,"5min":1.825,"15min":1.825},"last":1.0920000000000001}]},{"osd":5,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.089,"15min":1.1040000000000001},"min":{"1min":0.879,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.2909999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.143,"15min":1.1559999999999999},"min":{"1min":0.85099999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.409,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.1899999999999999}]},{"osd":6,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1339999999999999,"15min":1.1319999999999999},"min":{"1min":0.90800000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.425,"5min":1.734,"15min":1.734},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1639999999999999,"15min":1.1599999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.478,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.337}]},{"osd":7,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.1970000000000001,"15min":1.1930000000000001},"min":{"1min":1.0169999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.5920000000000001,"5min":1.861,"15min":1.861},"last":1.23},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.95199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.7210000000000001,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.123}]}]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302592,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069248,"statfs":{"total":96636764160,"available":96326909952,"internally_reserved":0,"allocated":1097728,"data_stored":800325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66100000000000003,"15min":0.67900000000000005},"min":{"1min":0.497,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82199999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.78000000000000003}]},{"osd":2,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.53500000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.2310000000000001,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.03},{"interface":"front","average":{"1min":0.75,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":0.90700000000000003,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.628}]},{"osd":3,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83399999999999996,"15min":0.82999999999999996},"min":{"1min":0.66000000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.90800000000000003,"15min":0.90400000000000003},"min":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.333,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.94199999999999995}]},{"osd":4,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.056,"15min":1.044},"min":{"1min":0.80000000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3899999999999999,"5min":1.919,"15min":1.919},"last":1.083},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.006,"15min":1.008},"min":{"1min":0.79800000000000004,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.96799999999999997}]},{"osd":5,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.022,"15min":1.0169999999999999},"min":{"1min":0.82299999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.141,"5min":1.1739999999999999,"15min":1.1990000000000001},"min":{"1min":0.84299999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4399999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.194,"15min":1.202},"min":{"1min":0.89600000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1619999999999999,"15min":1.1539999999999999},"min":{"1min":0.95199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.502,"5min":1.841,"15min":1.841},"last":1.1339999999999999}]},{"osd":7,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1459999999999999,"15min":1.1359999999999999},"min":{"1min":0.90600000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2509999999999999,"15min":1.2529999999999999},"min":{"1min":1.0369999999999999,"5min":0.878,"15min":0.878},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.88100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T18:01:59.655 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-08-15T17:58:22.996335+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=15, tm_hour=17, tm_min=58, tm_sec=22, tm_wday=3, tm_yday=228, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=15, tm_hour=18, tm_min=1, tm_sec=54, tm_wday=3, tm_yday=228, tm_isdst=0) 2024-08-15T18:01:59.655 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-15T18:02:05.575 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:02:05.578 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:02:19.657 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T18:02:19.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214b59e700 1 Processor -- start 2024-08-15T18:02:19.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214b59e700 1 -- start start 2024-08-15T18:02:19.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214b59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f214c159040 0x7f214c0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214b59e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f214c15d5b0 0x7f214c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214b59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214c0cf190 con 0x7f214c0d0010 2024-08-15T18:02:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214b59e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214c15d9f0 con 0x7f214c15d5b0 2024-08-15T18:02:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214b59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214c0d09a0 con 0x7f214c159040 2024-08-15T18:02:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214ad9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f214c15d5b0 0x7f214c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214ad9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f214c15d5b0 0x7f214c0d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33116/0 (socket says 172.21.15.38:33116) 2024-08-15T18:02:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214ad9d700 1 -- 172.21.15.38:0/953279179 learned_addr learned my addr 172.21.15.38:0/953279179 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:02:19.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214a59c700 1 --2- 172.21.15.38:0/953279179 >> v2:172.21.15.104:3300/0 conn(0x7f214c159040 0x7f214c0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:19.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214ad9d700 1 -- 172.21.15.38:0/953279179 >> v2:172.21.15.104:3300/0 conn(0x7f214c159040 msgr2=0x7f214c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:02:19.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214ad9d700 1 --2- 172.21.15.38:0/953279179 >> v2:172.21.15.104:3300/0 conn(0x7f214c159040 0x7f214c0cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:19.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214ad9d700 1 -- 172.21.15.38:0/953279179 >> v1:172.21.15.104:6789/0 conn(0x7f214c0d0010 legacy=0x7f214c15cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:19.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214ad9d700 1 -- 172.21.15.38:0/953279179 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c0ce5e0 con 0x7f214c15d5b0 2024-08-15T18:02:19.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214ad9d700 1 --2- 172.21.15.38:0/953279179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f214c15d5b0 0x7f214c0d0460 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f214c070d70 tx=0x7f2144009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1115e9a92723192 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:02:19.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214959a700 1 -- 172.21.15.38:0/953279179 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214400e070 con 0x7f214c15d5b0 2024-08-15T18:02:19.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214959a700 1 -- 172.21.15.38:0/953279179 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214400ba20 con 0x7f214c15d5b0 2024-08-15T18:02:19.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214959a700 1 -- 172.21.15.38:0/953279179 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21440092e0 con 0x7f214c15d5b0 2024-08-15T18:02:19.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214b59e700 1 -- 172.21.15.38:0/953279179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f214c15d5b0 msgr2=0x7f214c0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:19.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.769+0000 7f214b59e700 1 --2- 172.21.15.38:0/953279179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f214c15d5b0 0x7f214c0d0460 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f214c070d70 tx=0x7f2144009840 comp rx=0 tx=0).stop 2024-08-15T18:02:19.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.773+0000 7f214b59e700 1 -- 172.21.15.38:0/953279179 shutdown_connections 2024-08-15T18:02:19.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.773+0000 7f214b59e700 1 --2- 172.21.15.38:0/953279179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f214c15d5b0 0x7f214c0d0460 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:19.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.773+0000 7f214b59e700 1 --2- 172.21.15.38:0/953279179 >> v2:172.21.15.104:3300/0 conn(0x7f214c159040 0x7f214c0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:19.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.773+0000 7f214b59e700 1 -- 172.21.15.38:0/953279179 >> 172.21.15.38:0/953279179 conn(0x7f214c068250 msgr2=0x7f214c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:02:19.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.773+0000 7f214b59e700 1 -- 172.21.15.38:0/953279179 shutdown_connections 2024-08-15T18:02:19.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.773+0000 7f214b59e700 1 -- 172.21.15.38:0/953279179 wait complete. 2024-08-15T18:02:19.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.773+0000 7f214b59e700 1 Processor -- start 2024-08-15T18:02:19.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.773+0000 7f214b59e700 1 -- start start 2024-08-15T18:02:19.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214b59e700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f214c0d0010 0x7f214c0b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:19.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214b59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f214c159040 0x7f214c15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:19.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214b59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f214c15d5b0 0x7f214c16d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:19.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214b59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0b2900 con 0x7f214c159040 2024-08-15T18:02:19.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214b59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c15c4b0 con 0x7f214c15d5b0 2024-08-15T18:02:19.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214b59e700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214c14d560 con 0x7f214c0d0010 2024-08-15T18:02:19.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f2149d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f214c159040 0x7f214c15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:19.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f2149d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f214c159040 0x7f214c15bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33120/0 (socket says 172.21.15.38:33120) 2024-08-15T18:02:19.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f2149d9b700 1 -- 172.21.15.38:0/2984280405 learned_addr learned my addr 172.21.15.38:0/2984280405 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:02:19.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214a59c700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f214c0d0010 0x7f214c0b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:19.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214ad9d700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f214c15d5b0 0x7f214c16d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:19.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f2149d9b700 1 -- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f214c0d0010 msgr2=0x7f214c0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:19.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f2149d9b700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f214c0d0010 0x7f214c0b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:19.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f2149d9b700 1 -- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f214c15d5b0 msgr2=0x7f214c16d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:19.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f2149d9b700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f214c15d5b0 0x7f214c16d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:19.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f2149d9b700 1 -- 172.21.15.38:0/2984280405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2144009ae0 con 0x7f214c159040 2024-08-15T18:02:19.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214ad9d700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f214c15d5b0 0x7f214c16d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:02:19.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f2149d9b700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f214c159040 0x7f214c15bf70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f213400c4e0 tx=0x7f213400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:02:19.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f21427fc700 1 -- 172.21.15.38:0/2984280405 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134011590 con 0x7f214c159040 2024-08-15T18:02:19.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f21427fc700 1 -- 172.21.15.38:0/2984280405 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2134011710 con 0x7f214c159040 2024-08-15T18:02:19.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214b59e700 1 -- 172.21.15.38:0/2984280405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214c15b1b0 con 0x7f214c159040 2024-08-15T18:02:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f21427fc700 1 -- 172.21.15.38:0/2984280405 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134011ad0 con 0x7f214c159040 2024-08-15T18:02:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214b59e700 1 -- 172.21.15.38:0/2984280405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214c15ba50 con 0x7f214c159040 2024-08-15T18:02:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f214b59e700 1 -- 172.21.15.38:0/2984280405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214c15b3a0 con 0x7f214c159040 2024-08-15T18:02:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.777+0000 7f21427fc700 1 -- 172.21.15.38:0/2984280405 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f213401b3b0 con 0x7f214c159040 2024-08-15T18:02:19.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.781+0000 7f21427fc700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2130060860 0x7f2130062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:19.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.781+0000 7f21427fc700 1 -- 172.21.15.38:0/2984280405 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f213407be30 con 0x7f214c159040 2024-08-15T18:02:19.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.781+0000 7f214a59c700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2130060860 0x7f2130062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:19.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.781+0000 7f214a59c700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2130060860 0x7f2130062d10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f213c009d10 tx=0x7f213c0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:02:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.785+0000 7f21427fc700 1 -- 172.21.15.38:0/2984280405 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f213404c480 con 0x7f214c159040 2024-08-15T18:02:19.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.945+0000 7f214b59e700 1 -- 172.21.15.38:0/2984280405 --> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f214c15b600 con 0x7f2130060860 2024-08-15T18:02:19.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.953+0000 7f21427fc700 1 -- 172.21.15.38:0/2984280405 <== mgr.4108 v2:172.21.15.104:6832/24161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48846 (secure 0 0 0) 0x7f214c15b600 con 0x7f2130060860 2024-08-15T18:02:19.957 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:02:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.953+0000 7f214b59e700 1 -- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2130060860 msgr2=0x7f2130062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.953+0000 7f214b59e700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2130060860 0x7f2130062d10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f213c009d10 tx=0x7f213c0091a0 comp rx=0 tx=0).stop 2024-08-15T18:02:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.953+0000 7f214b59e700 1 -- 172.21.15.38:0/2984280405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f214c159040 msgr2=0x7f214c15bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.953+0000 7f214b59e700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f214c159040 0x7f214c15bf70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f213400c4e0 tx=0x7f213400c7f0 comp rx=0 tx=0).stop 2024-08-15T18:02:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.953+0000 7f214b59e700 1 -- 172.21.15.38:0/2984280405 shutdown_connections 2024-08-15T18:02:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.953+0000 7f214b59e700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f2130060860 0x7f2130062d10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.953+0000 7f214b59e700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f214c15d5b0 0x7f214c16d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.953+0000 7f214b59e700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f214c159040 0x7f214c15bf70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.953+0000 7f214b59e700 1 --2- 172.21.15.38:0/2984280405 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f214c0d0010 0x7f214c0b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.957+0000 7f214b59e700 1 -- 172.21.15.38:0/2984280405 >> 172.21.15.38:0/2984280405 conn(0x7f214c068250 msgr2=0x7f214c0b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:02:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.957+0000 7f214b59e700 1 -- 172.21.15.38:0/2984280405 shutdown_connections 2024-08-15T18:02:19.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:19.957+0000 7f214b59e700 1 -- 172.21.15.38:0/2984280405 wait complete. 2024-08-15T18:02:19.961 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-08-15T18:02:19.972 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-08-15T18:02:18.253122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2408904,"kb_used_data":5096,"kb_used_omap":13,"kb_used_meta":2403698,"kb_avail":752565816,"statfs":{"total":773094113280,"available":770627395584,"internally_reserved":0,"allocated":5218304,"data_stored":2860392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2461387456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003236"},"pg_stats":[{"pgid":"1.0","version":"43'119","reported_seq":196,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-08-15T18:02:02.645916+0000","last_change":"2024-08-15T18:02:02.645866+0000","last_active":"2024-08-15T18:02:02.645916+0000","last_peered":"2024-08-15T18:02:02.645916+0000","last_clean":"2024-08-15T18:02:02.645916+0000","last_became_active":"2024-08-15T17:58:24.391735+0000","last_became_peered":"2024-08-15T17:58:24.391735+0000","last_unstale":"2024-08-15T18:02:02.645916+0000","last_undegraded":"2024-08-15T18:02:02.645916+0000","last_fullsized":"2024-08-15T18:02:02.645916+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'119","last_scrub_stamp":"2024-08-15T18:02:02.645684+0000","last_deep_scrub":"43'119","last_deep_scrub_stamp":"2024-08-15T18:02:02.645684+0000","last_clean_scrub_stamp":"2024-08-15T18:02:02.645684+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:32:10.812837+0000","scrub_duration":0.0097752010000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302584,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069256,"statfs":{"total":96636764160,"available":96326918144,"internally_reserved":0,"allocated":1097728,"data_stored":800325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92600000000000005,"15min":0.92800000000000005},"min":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1919999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.86299999999999999,"15min":0.871},"min":{"1min":0.64300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.208,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.024}]},{"osd":1,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91800000000000004,"15min":0.90700000000000003},"min":{"1min":0.67300000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89200000000000002,"15min":0.875},"min":{"1min":0.76900000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0880000000000001,"5min":1.165,"15min":1.165},"last":1.1819999999999999}]},{"osd":2,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.73099999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.262,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1,"15min":1.022},"min":{"1min":0.65400000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.111,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.95099999999999996}]},{"osd":3,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.94599999999999995,"15min":0.93999999999999995},"min":{"1min":0.73799999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0880000000000001,"15min":1.083},"min":{"1min":0.86199999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3160000000000001,"5min":1.415,"15min":1.415},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.93999999999999995,"15min":0.96199999999999997},"min":{"1min":0.622,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.21,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.874},{"interface":"front","average":{"1min":1.034,"5min":1.0529999999999999,"15min":1.0780000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2350000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.2110000000000001}]},{"osd":5,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.105,"15min":1.1319999999999999},"min":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3080000000000001,"5min":1.599,"15min":1.599},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.131,"15min":1.1519999999999999},"min":{"1min":0.78400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3660000000000001,"5min":1.486,"15min":1.486},"last":1.25}]},{"osd":6,"last update":"Thu Aug 15 18:01:23 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1970000000000001,"15min":1.2170000000000001},"min":{"1min":0.85299999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.379,"5min":1.631,"15min":1.631},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1850000000000001,"15min":1.1950000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.49,"5min":1.518,"15min":1.518},"last":1.274}]}]},{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85199999999999998,"15min":0.81999999999999995},"min":{"1min":0.58999999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.224,"5min":1.226,"15min":1.226},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.91400000000000003,"15min":0.93600000000000005},"min":{"1min":0.622,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1970000000000001,"5min":1.516,"15min":1.516},"last":0.68500000000000005}]},{"osd":1,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88600000000000001,"15min":0.90800000000000003},"min":{"1min":0.66000000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.054,"5min":1.333,"15min":1.333},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.99299999999999999,"15min":1.006},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2669999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.96699999999999997}]},{"osd":2,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.98299999999999998,"15min":1.0009999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.274,"5min":1.369,"15min":1.369},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.0109999999999999,"15min":1.02},"min":{"1min":0.80600000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2609999999999999,"5min":1.409,"15min":1.409},"last":1.03}]},{"osd":3,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.67500000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1419999999999999,"5min":1.302,"15min":1.302},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.032,"5min":1.022,"15min":1.026},"min":{"1min":0.83599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.26,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.093}]},{"osd":4,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.95699999999999996,"15min":0.96599999999999997},"min":{"1min":0.63400000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.147,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.105,"15min":1.131},"min":{"1min":0.73299999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.306,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":1.1240000000000001}]},{"osd":5,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.876,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.234,"5min":1.345,"15min":1.345},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.161},"min":{"1min":0.999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.3240000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Thu Aug 15 18:01:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1970000000000001,"15min":1.232},"min":{"1min":0.97399999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.377,"5min":1.631,"15min":1.631},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.159,"15min":1.171},"min":{"1min":1,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4770000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.004}]}]},{"osd":4,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2310000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.95199999999999996,"15min":0.96699999999999997},"min":{"1min":0.65300000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2749999999999999,"5min":1.371,"15min":1.371},"last":1.179}]},{"osd":1,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.96699999999999997,"15min":0.98299999999999998},"min":{"1min":0.64300000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1060000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.98699999999999999,"15min":1.0309999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.167,"5min":1.708,"15min":1.708},"last":0.97999999999999998}]},{"osd":2,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.998,"15min":1.03},"min":{"1min":0.72799999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2629999999999999,"5min":1.514,"15min":1.514},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0429999999999999,"15min":1.0669999999999999},"min":{"1min":0.73099999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.2090000000000001,"5min":1.476,"15min":1.476},"last":0.90600000000000003}]},{"osd":3,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.1020000000000001,"15min":1.1399999999999999},"min":{"1min":0.82099999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.335,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":1.091,"15min":1.135},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3149999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.228}]},{"osd":5,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.012,"15min":1.054},"min":{"1min":0.68500000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.208,"5min":1.419,"15min":1.419},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.137,"15min":1.1990000000000001},"min":{"1min":0.69699999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.3120000000000001,"5min":1.621,"15min":1.621},"last":0.94299999999999995}]},{"osd":6,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.121,"15min":1.167},"min":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3240000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.198,"15min":1.264},"min":{"1min":0.82899999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.2629999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.872}]},{"osd":7,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2789999999999999,"15min":1.353},"min":{"1min":0.86799999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3799999999999999,"5min":1.772,"15min":1.772},"last":1.054},{"interface":"front","average":{"1min":1.171,"5min":1.2989999999999999,"15min":1.345},"min":{"1min":0.91700000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.391,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.143}]}]},{"osd":5,"up_from":14,"seq":60129542195,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.98199999999999998,"15min":0.96599999999999997},"min":{"1min":0.71099999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.248,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.04,"15min":1.042},"min":{"1min":0.75600000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2470000000000001,"5min":1.383,"15min":1.383},"last":1.0589999999999999}]},{"osd":1,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.66700000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96499999999999997,"15min":0.95699999999999996},"min":{"1min":0.71699999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2070000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95399999999999996,"15min":0.96099999999999997},"min":{"1min":0.76700000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.177,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.996,"15min":1.012},"min":{"1min":0.69699999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.2949999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.92400000000000004,"15min":0.92500000000000004},"min":{"1min":0.73399999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.01,"15min":0.98799999999999999},"min":{"1min":0.80900000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.181}]},{"osd":4,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.94099999999999995,"15min":0.94799999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.036,"5min":1.0189999999999999,"15min":1.0249999999999999},"min":{"1min":0.73099999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.93100000000000005}]},{"osd":6,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.99199999999999999,"15min":0.99399999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1950000000000001,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.117,"15min":1.1080000000000001},"min":{"1min":0.997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.26}]},{"osd":7,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.071,"15min":1.075},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.028},{"interface":"front","average":{"1min":1.198,"5min":1.1679999999999999,"15min":1.165},"min":{"1min":0.997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.3420000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.135}]}]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.998,"15min":0.998},"last":1.179},{"interface":"front","average":{"1min":0.748,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.99299999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.53000000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.03,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.69599999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.98699999999999999}]},{"osd":2,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.872,"15min":0.88200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.214,"5min":1.74,"15min":1.74},"last":0.871},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.97899999999999998,"15min":0.99299999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.089,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.0960000000000001}]},{"osd":4,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1100000000000001,"15min":1.1319999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.444,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.022,"15min":1.0309999999999999},"min":{"1min":0.83599999999999997,"5min":0.755,"15min":0.755},"max":{"1min":1.4239999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.89000000000000001}]},{"osd":5,"last update":"Thu Aug 15 18:01:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.167,"15min":1.181},"min":{"1min":0.76700000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5649999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.028,"5min":1.1220000000000001,"15min":1.1699999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.448,"5min":1.581,"15min":1.581},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1819999999999999,"15min":1.204},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5529999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.704},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6160000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.83099999999999996}]},{"osd":7,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2030000000000001,"15min":1.2250000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5589999999999999,"5min":2.073,"15min":2.073},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2250000000000001,"15min":1.244},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.4890000000000001,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.806}]}]},{"osd":2,"up_from":14,"seq":60129542194,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302016,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069824,"statfs":{"total":96636764160,"available":96327499776,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.105,"5min":1.204,"15min":1.204},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.61099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0589999999999999,"5min":1.111,"15min":1.111},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.626,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1140000000000001,"5min":1.127,"15min":1.127},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2270000000000001,"5min":1.278,"15min":1.278},"last":0.80600000000000005}]},{"osd":3,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.57899999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0009999999999999,"5min":1.073,"15min":1.073},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.93000000000000005},"min":{"1min":0.73699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2090000000000001,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.95899999999999996}]},{"osd":4,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.0089999999999999,"15min":0.996},"min":{"1min":0.77500000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.296,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.085,"5min":1.083,"15min":1.0900000000000001},"min":{"1min":0.82599999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.335,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.1180000000000001}]},{"osd":5,"last update":"Thu Aug 15 18:01:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.1359999999999999,"15min":1.155},"min":{"1min":0.85499999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.292,"5min":1.573,"15min":1.573},"last":0.997},{"interface":"front","average":{"1min":1.091,"5min":1.099,"15min":1.107},"min":{"1min":0.87,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3300000000000001,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.92000000000000004}]},{"osd":6,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.137,"15min":1.159},"min":{"1min":0.81399999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.3480000000000001,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.1739999999999999,"15min":1.1830000000000001},"min":{"1min":0.84499999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.399,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.091}]},{"osd":7,"last update":"Thu Aug 15 18:01:32 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2110000000000001,"15min":1.218},"min":{"1min":0.96499999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.173},{"interface":"front","average":{"1min":1.202,"5min":1.2290000000000001,"15min":1.244},"min":{"1min":1.0129999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4259999999999999,"5min":1.861,"15min":1.861},"last":1.1499999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069832,"statfs":{"total":96636764160,"available":96327507968,"internally_reserved":0,"allocated":503808,"data_stored":209957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.52300000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.94299999999999995}]},{"osd":2,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.443,"15min":0.443},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90600000000000003,"15min":0.89100000000000001},"min":{"1min":0.70699999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90100000000000002,"15min":0.92400000000000004},"min":{"1min":0.66300000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0700000000000001,"5min":1.778,"15min":1.778},"last":1.0900000000000001}]},{"osd":4,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.069,"15min":1.093},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4199999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0529999999999999,"15min":1.056},"min":{"1min":0.79200000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.4750000000000001,"5min":1.825,"15min":1.825},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.089,"15min":1.1040000000000001},"min":{"1min":0.879,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.2909999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.143,"15min":1.1559999999999999},"min":{"1min":0.85099999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.409,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.85399999999999998}]},{"osd":6,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1339999999999999,"15min":1.1319999999999999},"min":{"1min":0.90800000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.425,"5min":1.734,"15min":1.734},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1639999999999999,"15min":1.1599999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.478,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Aug 15 18:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.1970000000000001,"15min":1.1930000000000001},"min":{"1min":1.0169999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.5920000000000001,"5min":1.861,"15min":1.861},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.214,"15min":1.22},"min":{"1min":0.95199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.7210000000000001,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.7909999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542195,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302584,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069256,"statfs":{"total":96636764160,"available":96326918144,"internally_reserved":0,"allocated":1097728,"data_stored":800325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66100000000000003,"15min":0.67900000000000005},"min":{"1min":0.497,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82199999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.1339999999999999}]},{"osd":2,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.53500000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.2310000000000001,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":0.90700000000000003,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.871}]},{"osd":3,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83399999999999996,"15min":0.82999999999999996},"min":{"1min":0.66000000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.90800000000000003,"15min":0.90400000000000003},"min":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.333,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.331}]},{"osd":4,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.056,"15min":1.044},"min":{"1min":0.80000000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3899999999999999,"5min":1.919,"15min":1.919},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.006,"15min":1.008},"min":{"1min":0.79800000000000004,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.0940000000000001}]},{"osd":5,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.022,"15min":1.0169999999999999},"min":{"1min":0.82299999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.141,"5min":1.1739999999999999,"15min":1.1990000000000001},"min":{"1min":0.84299999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4399999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.4139999999999999}]},{"osd":6,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.194,"15min":1.202},"min":{"1min":0.89600000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":1.3},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1619999999999999,"15min":1.1539999999999999},"min":{"1min":0.95199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.502,"5min":1.841,"15min":1.841},"last":1.7709999999999999}]},{"osd":7,"last update":"Thu Aug 15 18:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1459999999999999,"15min":1.1359999999999999},"min":{"1min":0.90600000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.385},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2509999999999999,"15min":1.2529999999999999},"min":{"1min":1.0369999999999999,"5min":0.878,"15min":0.878},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.3680000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T18:02:19.974 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-08-15T18:02:20.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ffa01176700 1 Processor -- start 2024-08-15T18:02:20.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ffa01176700 1 -- start start 2024-08-15T18:02:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ffa01176700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff9fc1514d0 0x7ff9fc06ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ffa01176700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9fc06e010 con 0x7ff9fc1574f0 2024-08-15T18:02:20.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ffa01176700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9fc06efa0 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ffa01176700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9fc157fb0 con 0x7ff9fc1514d0 2024-08-15T18:02:20.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9fbfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff9fc1514d0 0x7ff9fc06ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:20.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9fb7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff9fc1574f0 0x7ff9fc0b0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53032/0 (socket says 172.21.15.38:53032) 2024-08-15T18:02:20.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9fb7fe700 1 -- 172.21.15.38:0/1844865970 learned_addr learned my addr 172.21.15.38:0/1844865970 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:02:20.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9faffd700 1 -- 172.21.15.38:0/1844865970 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210510700 0 0) 0x7ff9fc06e010 con 0x7ff9fc1574f0 2024-08-15T18:02:20.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9faffd700 1 -- 172.21.15.38:0/1844865970 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e8003570 con 0x7ff9fc1574f0 2024-08-15T18:02:20.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9fbfff700 1 -- 172.21.15.38:0/1844865970 >> v1:172.21.15.104:6789/0 conn(0x7ff9fc0b14a0 legacy=0x7ff9fc0c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9fbfff700 1 -- 172.21.15.38:0/1844865970 >> v1:172.21.15.38:6789/0 conn(0x7ff9fc1574f0 legacy=0x7ff9fc0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9fbfff700 1 -- 172.21.15.38:0/1844865970 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc158290 con 0x7ff9fc1514d0 2024-08-15T18:02:20.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9fbfff700 1 --2- 172.21.15.38:0/1844865970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9fc1514d0 0x7ff9fc06ea60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0098e0 tx=0x7ff9ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fb14bd6032b991 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:02:20.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9faffd700 1 -- 172.21.15.38:0/1844865970 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00b640 con 0x7ff9fc1514d0 2024-08-15T18:02:20.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9faffd700 1 -- 172.21.15.38:0/1844865970 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec00b7a0 con 0x7ff9fc1514d0 2024-08-15T18:02:20.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.073+0000 7ff9faffd700 1 -- 172.21.15.38:0/1844865970 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00bb40 con 0x7ff9fc1514d0 2024-08-15T18:02:20.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 -- 172.21.15.38:0/1844865970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9fc1514d0 msgr2=0x7ff9fc06ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 --2- 172.21.15.38:0/1844865970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9fc1514d0 0x7ff9fc06ea60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0098e0 tx=0x7ff9ec0091a0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 -- 172.21.15.38:0/1844865970 shutdown_connections 2024-08-15T18:02:20.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 --2- 172.21.15.38:0/1844865970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9fc1514d0 0x7ff9fc06ea60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 -- 172.21.15.38:0/1844865970 >> 172.21.15.38:0/1844865970 conn(0x7ff9fc0678a0 msgr2=0x7ff9fc067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:02:20.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 -- 172.21.15.38:0/1844865970 shutdown_connections 2024-08-15T18:02:20.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 -- 172.21.15.38:0/1844865970 wait complete. 2024-08-15T18:02:20.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 Processor -- start 2024-08-15T18:02:20.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 -- start start 2024-08-15T18:02:20.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9fc0b14a0 0x7ff9fc168320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:20.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9fc1514d0 0x7ff9fc16a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:20.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc1574f0 0x7ff9fc167c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:20.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc16a570 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc154090 con 0x7ff9fc1574f0 2024-08-15T18:02:20.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ffa01176700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc168150 con 0x7ff9fc1514d0 2024-08-15T18:02:20.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ff9fbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9fc0b14a0 0x7ff9fc168320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:20.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ff9fbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9fc0b14a0 0x7ff9fc168320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33138/0 (socket says 172.21.15.38:33138) 2024-08-15T18:02:20.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.077+0000 7ff9fbfff700 1 -- 172.21.15.38:0/3228207366 learned_addr learned my addr 172.21.15.38:0/3228207366 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:02:20.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ffa00975700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc1574f0 0x7ff9fc167c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:20.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ff9fb7fe700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9fc1514d0 0x7ff9fc16a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:20.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ff9fbfff700 1 -- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9fc1514d0 msgr2=0x7ff9fc16a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ff9fbfff700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9fc1514d0 0x7ff9fc16a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ff9fbfff700 1 -- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc1574f0 msgr2=0x7ff9fc167c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ff9fbfff700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc1574f0 0x7ff9fc167c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ff9fbfff700 1 -- 172.21.15.38:0/3228207366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec009420 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ff9fb7fe700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9fc1514d0 0x7ff9fc16a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T18:02:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ff9fbfff700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9fc0b14a0 0x7ff9fc168320 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0093f0 tx=0x7ff9ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:02:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ff9f3fff700 1 -- 172.21.15.38:0/3228207366 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00bb20 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ff9f3fff700 1 -- 172.21.15.38:0/3228207366 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec011420 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ffa01176700 1 -- 172.21.15.38:0/3228207366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc153480 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ff9f3fff700 1 -- 172.21.15.38:0/3228207366 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec011800 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ffa01176700 1 -- 172.21.15.38:0/3228207366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc153a40 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.081+0000 7ffa01176700 1 -- 172.21.15.38:0/3228207366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc064330 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.085+0000 7ff9f3fff700 1 -- 172.21.15.38:0/3228207366 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff9ec011c30 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.085+0000 7ff9f3fff700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9dc060860 0x7ff9dc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.085+0000 7ff9f3fff700 1 -- 172.21.15.38:0/3228207366 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7ff9ec07e2d0 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.085+0000 7ff9fb7fe700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9dc060860 0x7ff9dc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.085+0000 7ff9fb7fe700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9dc060860 0x7ff9dc062d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc06ba10 tx=0x7ff9e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:02:20.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.093+0000 7ff9f3fff700 1 -- 172.21.15.38:0/3228207366 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff9ec04e920 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.257+0000 7ffa01176700 1 -- 172.21.15.38:0/3228207366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7ff9fc14ff60 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.269+0000 7ff9f3fff700 1 -- 172.21.15.38:0/3228207366 <== mon.0 v2:172.21.15.38:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff9ec07e110 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.269+0000 7ff9f3fff700 1 -- 172.21.15.38:0/3228207366 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7ff9ec051f40 con 0x7ff9fc0b14a0 2024-08-15T18:02:20.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 -- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9dc060860 msgr2=0x7ff9dc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9dc060860 0x7ff9dc062d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc06ba10 tx=0x7ff9e4009040 comp rx=0 tx=0).stop 2024-08-15T18:02:20.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 -- 172.21.15.38:0/3228207366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9fc0b14a0 msgr2=0x7ff9fc168320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9fc0b14a0 0x7ff9fc168320 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0093f0 tx=0x7ff9ec017000 comp rx=0 tx=0).stop 2024-08-15T18:02:20.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 -- 172.21.15.38:0/3228207366 shutdown_connections 2024-08-15T18:02:20.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7ff9dc060860 0x7ff9dc062d10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc1574f0 0x7ff9fc167c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9fc1514d0 0x7ff9fc16a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 --2- 172.21.15.38:0/3228207366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9fc0b14a0 0x7ff9fc168320 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 -- 172.21.15.38:0/3228207366 >> 172.21.15.38:0/3228207366 conn(0x7ff9fc0678a0 msgr2=0x7ff9fc159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:02:20.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 -- 172.21.15.38:0/3228207366 shutdown_connections 2024-08-15T18:02:20.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.273+0000 7ffa01176700 1 -- 172.21.15.38:0/3228207366 wait complete. 2024-08-15T18:02:20.291 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-15T18:02:20.291 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T18:02:20.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635ffff700 1 Processor -- start 2024-08-15T18:02:20.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635ffff700 1 -- start start 2024-08-15T18:02:20.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f63601428e0 0x7f6360068af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:20.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635ffff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f636015cd10 0x7f63600b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:20.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635ffff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6360069030 con 0x7f63600b1220 2024-08-15T18:02:20.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635ffff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6360069ee0 con 0x7f63601428e0 2024-08-15T18:02:20.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635ffff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63600b16a0 con 0x7f636015cd10 2024-08-15T18:02:20.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635f7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f63600b1220 0x7f63600b2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53046/0 (socket says 172.21.15.38:53046) 2024-08-15T18:02:20.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635f7fe700 1 -- 172.21.15.38:0/901902128 learned_addr learned my addr 172.21.15.38:0/901902128 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-08-15T18:02:20.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635effd700 1 --2- 172.21.15.38:0/901902128 >> v2:172.21.15.104:3300/0 conn(0x7f63601428e0 0x7f6360068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:20.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f63567fc700 1 --2- 172.21.15.38:0/901902128 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f636015cd10 0x7f63600b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:20.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635e7fc700 1 -- 172.21.15.38:0/901902128 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107030612 0 0) 0x7f6360069030 con 0x7f63600b1220 2024-08-15T18:02:20.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635e7fc700 1 -- 172.21.15.38:0/901902128 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f634c003570 con 0x7f63600b1220 2024-08-15T18:02:20.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635effd700 1 -- 172.21.15.38:0/901902128 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f636015cd10 msgr2=0x7f63600b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T18:02:20.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635effd700 1 --2- 172.21.15.38:0/901902128 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f636015cd10 0x7f63600b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635effd700 1 -- 172.21.15.38:0/901902128 >> v1:172.21.15.38:6789/0 conn(0x7f63600b1220 legacy=0x7f63600b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f635effd700 1 -- 172.21.15.38:0/901902128 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6360069260 con 0x7f63601428e0 2024-08-15T18:02:20.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.401+0000 7f63567fc700 1 --2- 172.21.15.38:0/901902128 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f636015cd10 0x7f63600b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T18:02:20.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635effd700 1 --2- 172.21.15.38:0/901902128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63601428e0 0x7f6360068af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6350009910 tx=0x7f63500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9321a85bc0e4210f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:02:20.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635e7fc700 1 -- 172.21.15.38:0/901902128 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350011630 con 0x7f63601428e0 2024-08-15T18:02:20.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635e7fc700 1 -- 172.21.15.38:0/901902128 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6350011790 con 0x7f63601428e0 2024-08-15T18:02:20.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635e7fc700 1 -- 172.21.15.38:0/901902128 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350011b50 con 0x7f63601428e0 2024-08-15T18:02:20.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635ffff700 1 -- 172.21.15.38:0/901902128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63601428e0 msgr2=0x7f6360068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635ffff700 1 --2- 172.21.15.38:0/901902128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63601428e0 0x7f6360068af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6350009910 tx=0x7f63500091a0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635ffff700 1 -- 172.21.15.38:0/901902128 shutdown_connections 2024-08-15T18:02:20.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635ffff700 1 --2- 172.21.15.38:0/901902128 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f636015cd10 0x7f63600b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635ffff700 1 --2- 172.21.15.38:0/901902128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63601428e0 0x7f6360068af0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635ffff700 1 -- 172.21.15.38:0/901902128 >> 172.21.15.38:0/901902128 conn(0x7f63600ce990 msgr2=0x7f636006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:02:20.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635ffff700 1 -- 172.21.15.38:0/901902128 shutdown_connections 2024-08-15T18:02:20.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635ffff700 1 -- 172.21.15.38:0/901902128 wait complete. 2024-08-15T18:02:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635ffff700 1 Processor -- start 2024-08-15T18:02:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.405+0000 7f635ffff700 1 -- start start 2024-08-15T18:02:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635ffff700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f63600b1220 0x7f63600a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63601428e0 0x7f636015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f636015cd10 0x7f63600c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635ffff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6360169fc0 con 0x7f636015cd10 2024-08-15T18:02:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635ffff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636016ae20 con 0x7f63601428e0 2024-08-15T18:02:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635ffff700 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636015c880 con 0x7f63600b1220 2024-08-15T18:02:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f636015cd10 0x7f63600c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f63567fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63601428e0 0x7f636015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f636015cd10 0x7f63600c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33144/0 (socket says 172.21.15.38:33144) 2024-08-15T18:02:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635f7fe700 1 -- 172.21.15.38:0/2610004814 learned_addr learned my addr 172.21.15.38:0/2610004814 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-08-15T18:02:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635effd700 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f63600b1220 0x7f63600a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:20.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635f7fe700 1 -- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f63600b1220 msgr2=0x7f63600a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635f7fe700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f63600b1220 0x7f63600a9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635f7fe700 1 -- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63601428e0 msgr2=0x7f636015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635f7fe700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63601428e0 0x7f636015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635f7fe700 1 -- 172.21.15.38:0/2610004814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348009420 con 0x7f636015cd10 2024-08-15T18:02:20.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635f7fe700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f636015cd10 0x7f63600c9230 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f635800cc40 tx=0x7f6358001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:02:20.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f63577fe700 1 -- 172.21.15.38:0/2610004814 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800a830 con 0x7f636015cd10 2024-08-15T18:02:20.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f63577fe700 1 -- 172.21.15.38:0/2610004814 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f635800a990 con 0x7f636015cd10 2024-08-15T18:02:20.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635ffff700 1 -- 172.21.15.38:0/2610004814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6350009420 con 0x7f636015cd10 2024-08-15T18:02:20.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f63577fe700 1 -- 172.21.15.38:0/2610004814 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800ad10 con 0x7f636015cd10 2024-08-15T18:02:20.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.409+0000 7f635ffff700 1 -- 172.21.15.38:0/2610004814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636016a570 con 0x7f636015cd10 2024-08-15T18:02:20.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.413+0000 7f635ffff700 1 -- 172.21.15.38:0/2610004814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636016afb0 con 0x7f636015cd10 2024-08-15T18:02:20.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.413+0000 7f63577fe700 1 -- 172.21.15.38:0/2610004814 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6358007300 con 0x7f636015cd10 2024-08-15T18:02:20.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.413+0000 7f63577fe700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6340060860 0x7f6340062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T18:02:20.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.413+0000 7f63577fe700 1 -- 172.21.15.38:0/2610004814 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f635807cbf0 con 0x7f636015cd10 2024-08-15T18:02:20.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.413+0000 7f635effd700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6340060860 0x7f6340062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T18:02:20.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.413+0000 7f635effd700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6340060860 0x7f6340062d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f63600c9d60 tx=0x7f63500091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T18:02:20.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.421+0000 7f63577fe700 1 -- 172.21.15.38:0/2610004814 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f635804d3b0 con 0x7f636015cd10 2024-08-15T18:02:20.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f635ffff700 1 -- 172.21.15.38:0/2610004814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63600638f0 con 0x7f636015cd10 2024-08-15T18:02:20.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f63577fe700 1 -- 172.21.15.38:0/2610004814 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f63580509d0 con 0x7f636015cd10 2024-08-15T18:02:20.629 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-08-15T18:02:20.629 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/373633111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":373633111},{"type":"v1","addr":"172.21.15.38:6833","nonce":373633111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/389886189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":389886189},{"type":"v1","addr":"172.21.15.38:6835","nonce":389886189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3362365103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3362365103},{"type":"v1","addr":"172.21.15.104:6837","nonce":3362365103}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3229467722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3229467722},{"type":"v1","addr":"172.21.15.104:6835","nonce":3229467722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[]} 2024-08-15T18:02:20.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f635ffff700 1 -- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6340060860 msgr2=0x7f6340062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f635ffff700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6340060860 0x7f6340062d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f63600c9d60 tx=0x7f63500091a0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f635ffff700 1 -- 172.21.15.38:0/2610004814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f636015cd10 msgr2=0x7f63600c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T18:02:20.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f635ffff700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f636015cd10 0x7f63600c9230 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f635800cc40 tx=0x7f6358001a40 comp rx=0 tx=0).stop 2024-08-15T18:02:20.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f635ffff700 1 -- 172.21.15.38:0/2610004814 shutdown_connections 2024-08-15T18:02:20.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f635ffff700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:6832/24161,v1:172.21.15.104:6833/24161] conn(0x7f6340060860 0x7f6340062d10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f635ffff700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f636015cd10 0x7f63600c9230 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f635ffff700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63601428e0 0x7f636015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f635ffff700 1 --2- 172.21.15.38:0/2610004814 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f63600b1220 0x7f63600a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T18:02:20.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.625+0000 7f635ffff700 1 -- 172.21.15.38:0/2610004814 >> 172.21.15.38:0/2610004814 conn(0x7f63600ce990 msgr2=0x7f63600afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T18:02:20.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.629+0000 7f635ffff700 1 -- 172.21.15.38:0/2610004814 shutdown_connections 2024-08-15T18:02:20.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-08-15T18:02:20.629+0000 7f635ffff700 1 -- 172.21.15.38:0/2610004814 wait complete. 2024-08-15T18:02:20.633 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 22 2024-08-15T18:02:20.643 DEBUG:tasks.cephfs.filesystem:already dead... 2024-08-15T18:02:20.643 INFO:teuthology.misc:Shutting down mds daemons... 2024-08-15T18:02:20.643 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-08-15T18:02:20.643 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:20.658 INFO:tasks.ceph.mds.a:Stopped 2024-08-15T18:02:20.658 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-08-15T18:02:20.658 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:20.671 INFO:tasks.ceph.mds.c:Stopped 2024-08-15T18:02:20.671 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-08-15T18:02:20.671 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:20.688 INFO:tasks.ceph.mds.b:Stopped 2024-08-15T18:02:20.688 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-15T18:02:20.688 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:20.704 INFO:tasks.ceph.mds.d:Stopped 2024-08-15T18:02:20.704 INFO:teuthology.misc:Shutting down osd daemons... 2024-08-15T18:02:20.705 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-08-15T18:02:20.705 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:20.772 INFO:tasks.ceph.osd.0:Stopped 2024-08-15T18:02:20.773 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-08-15T18:02:20.773 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:20.846 INFO:tasks.ceph.osd.1:Stopped 2024-08-15T18:02:20.847 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-08-15T18:02:20.847 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:20.911 INFO:tasks.ceph.osd.2:Stopped 2024-08-15T18:02:20.911 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-08-15T18:02:20.911 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:20.979 INFO:tasks.ceph.osd.3:Stopped 2024-08-15T18:02:20.980 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-08-15T18:02:20.980 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:21.055 INFO:tasks.ceph.osd.4:Stopped 2024-08-15T18:02:21.056 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-08-15T18:02:21.056 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:21.127 INFO:tasks.ceph.osd.5:Stopped 2024-08-15T18:02:21.128 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-08-15T18:02:21.128 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:21.196 INFO:tasks.ceph.osd.6:Stopped 2024-08-15T18:02:21.197 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-08-15T18:02:21.197 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:21.282 INFO:tasks.ceph.osd.7:Stopped 2024-08-15T18:02:21.282 INFO:teuthology.misc:Shutting down mgr daemons... 2024-08-15T18:02:21.283 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-08-15T18:02:21.283 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:21.308 INFO:tasks.ceph.mgr.y:Stopped 2024-08-15T18:02:21.309 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-08-15T18:02:21.309 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:21.337 INFO:tasks.ceph.mgr.x:Stopped 2024-08-15T18:02:21.337 INFO:teuthology.misc:Shutting down mon daemons... 2024-08-15T18:02:21.338 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-08-15T18:02:21.338 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:21.357 INFO:tasks.ceph.mon.a:Stopped 2024-08-15T18:02:21.357 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-08-15T18:02:21.357 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:21.374 INFO:tasks.ceph.mon.b:Stopped 2024-08-15T18:02:21.374 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-08-15T18:02:21.374 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T18:02:21.435 INFO:tasks.ceph.mon.c:Stopped 2024-08-15T18:02:21.435 INFO:tasks.ceph:Checking cluster log for badness... 2024-08-15T18:02:21.435 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-08-15T18:02:21.459 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T18:02:21.459 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-08-15T18:02:21.752 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T18:02:21.753 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-08-15T18:02:21.936 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T18:02:21.936 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-08-15T18:02:22.095 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T18:02:22.095 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-08-15T18:02:22.269 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T18:02:22.270 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-08-15T18:02:22.501 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T18:02:22.502 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-08-15T18:02:22.740 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T18:02:22.741 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-08-15T18:02:22.948 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T18:02:22.948 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-08-15T18:02:23.177 INFO:tasks.ceph:Archiving mon data... 2024-08-15T18:02:23.179 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038/data/mon.a.tgz 2024-08-15T18:02:23.179 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2024-08-15T18:02:23.186 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.Vu5eXdvpNA 2024-08-15T18:02:23.186 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Vu5eXdvpNA 2024-08-15T18:02:23.272 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.Vu5eXdvpNA 2024-08-15T18:02:23.338 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.Vu5eXdvpNA is 305KB 2024-08-15T18:02:23.358 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.Vu5eXdvpNA 2024-08-15T18:02:23.364 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038/data/mon.b.tgz 2024-08-15T18:02:23.364 DEBUG:teuthology.orchestra.run.smithi104:> mktemp 2024-08-15T18:02:23.371 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.43ylP7wJZR 2024-08-15T18:02:23.371 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.43ylP7wJZR 2024-08-15T18:02:23.519 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.43ylP7wJZR 2024-08-15T18:02:23.540 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.43ylP7wJZR is 405KB 2024-08-15T18:02:23.604 DEBUG:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.43ylP7wJZR 2024-08-15T18:02:23.610 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038/data/mon.c.tgz 2024-08-15T18:02:23.611 DEBUG:teuthology.orchestra.run.smithi104:> mktemp 2024-08-15T18:02:23.617 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.GyzPlc7kbs 2024-08-15T18:02:23.617 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.GyzPlc7kbs 2024-08-15T18:02:23.762 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.GyzPlc7kbs 2024-08-15T18:02:23.785 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.GyzPlc7kbs is 405KB 2024-08-15T18:02:23.852 DEBUG:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.GyzPlc7kbs 2024-08-15T18:02:23.858 INFO:tasks.ceph:Cleaning ceph cluster... 2024-08-15T18:02:23.859 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-15T18:02:23.861 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-15T18:02:23.875 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-08-15T18:02:23.875 DEBUG:teuthology.orchestra.run.smithi038:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-15T18:02:23.877 DEBUG:teuthology.orchestra.run.smithi104:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-15T18:02:23.909 INFO:teuthology.orchestra.run.smithi038.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-15T18:02:23.946 INFO:teuthology.orchestra.run.smithi104.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-15T18:02:23.952 INFO:tasks.ceph:Archiving crash dumps... 2024-08-15T18:02:23.954 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038/remote/smithi038/crash 2024-08-15T18:02:23.955 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-15T18:02:23.970 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038/remote/smithi104/crash 2024-08-15T18:02:23.971 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-15T18:02:23.988 INFO:tasks.ceph:Shutting down logrotate 2024-08-15T18:02:23.988 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:02:24.016 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T18:02:24.075 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-15T18:02:24.092 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-15T18:02:24.107 INFO:tasks.ceph:Compressing logs... 2024-08-15T18:02:24.108 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-15T18:02:24.110 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-15T18:02:24.129 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24298.log 2024-08-15T18:02:24.129 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24273.log 2024-08-15T18:02:24.130 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-08-15T18:02:24.130 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.24298.log: /var/log/ceph/ceph-client.admin.24273.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-08-15T18:02:24.131 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mds.d.log: 88.3% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.24273.log.gz -- replaced with /var/log/ceph/ceph-client.admin.24298.log.gz 2024-08-15T18:02:24.131 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-08-15T18:02:24.131 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24377.log 2024-08-15T18:02:24.132 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.30055.log 2024-08-15T18:02:24.132 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24459.log 2024-08-15T18:02:24.133 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.24377.log: /var/log/ceph/ceph-client.3.30055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.29941.log 2024-08-15T18:02:24.133 INFO:teuthology.orchestra.run.smithi104.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.24377.log.gz 2024-08-15T18:02:24.133 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.24459.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-08-15T18:02:24.134 INFO:teuthology.orchestra.run.smithi104.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.24459.log.gz 2024-08-15T18:02:24.134 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-08-15T18:02:24.135 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.29941.log: 88.6% -- replaced with /var/log/ceph/ceph-client.3.30055.log.gz 2024-08-15T18:02:24.135 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.5.log: 89.3% -- replaced with /var/log/ceph/ceph-client.1.29941.log.gz 2024-08-15T18:02:24.136 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-08-15T18:02:24.136 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24404.log 2024-08-15T18:02:24.138 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29392.log 2024-08-15T18:02:24.148 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-15T18:02:24.149 INFO:teuthology.orchestra.run.smithi104.stderr:gzip /var/log/ceph/ceph-client.admin.24404.log: -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-08-15T18:02:24.149 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph.audit.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.24404.log.gz 2024-08-15T18:02:24.149 INFO:teuthology.orchestra.run.smithi104.stderr: 93.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-15T18:02:24.149 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24350.log 2024-08-15T18:02:24.150 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-08-15T18:02:24.150 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.24350.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.24350.log.gz 2024-08-15T18:02:24.155 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-15T18:02:24.156 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-08-15T18:02:24.156 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph.log: 91.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-15T18:02:24.156 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.30232.log 2024-08-15T18:02:24.184 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.4.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-08-15T18:02:24.184 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24432.log 2024-08-15T18:02:24.184 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.30232.log: 95.3% -- replaced with /var/log/ceph/ceph-client.1.30232.log.gz 2024-08-15T18:02:24.184 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24323.log 2024-08-15T18:02:24.185 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.24432.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.24432.log.gz 2024-08-15T18:02:24.185 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-08-15T18:02:24.185 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.24323.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.24323.log.gz 2024-08-15T18:02:24.189 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.7.log: 91.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-08-15T18:02:24.197 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-08-15T18:02:24.235 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34538.log 2024-08-15T18:02:24.235 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30532.log 2024-08-15T18:02:24.235 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31843.log 2024-08-15T18:02:24.236 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34538.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29392.log.gz 2024-08-15T18:02:24.236 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29448.log 2024-08-15T18:02:24.236 INFO:teuthology.orchestra.run.smithi038.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34538.log.gz 2024-08-15T18:02:24.236 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30532.log.gz 2024-08-15T18:02:24.236 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32019.log 2024-08-15T18:02:24.236 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30043.log 2024-08-15T18:02:24.236 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30640.log 2024-08-15T18:02:24.237 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31843.log.gz 2024-08-15T18:02:24.237 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32019.log: /var/log/ceph/ceph-client.admin.30043.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29448.log.gz 2024-08-15T18:02:24.237 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33388.log 2024-08-15T18:02:24.237 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32019.log.gz 2024-08-15T18:02:24.237 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 /var/log/ceph/ceph-client.admin.30640.log: --verbose -- 88.0%/var/log/ceph/ceph-client.admin.24258.log -- replaced with /var/log/ceph/ceph-client.admin.30043.log.gz 2024-08-15T18:02:24.237 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-08-15T18:02:24.237 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30640.log.gz 2024-08-15T18:02:24.237 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30036.log 2024-08-15T18:02:24.238 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.24258.log: -- /var/log/ceph/ceph-mon.a.log 2024-08-15T18:02:24.238 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33388.log.gz 2024-08-15T18:02:24.238 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33658.log 2024-08-15T18:02:24.238 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.24258.log.gz 2024-08-15T18:02:24.238 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30036.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31542.log 2024-08-15T18:02:24.238 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30036.log.gz 2024-08-15T18:02:24.238 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30613.log 2024-08-15T18:02:24.238 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33658.log: gzip /var/log/ceph/ceph-client.admin.31542.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29695.log 2024-08-15T18:02:24.239 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33658.log.gz 2024-08-15T18:02:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34511.log 2024-08-15T18:02:24.239 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31542.log.gz 2024-08-15T18:02:24.239 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30613.log.gz 2024-08-15T18:02:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33550.log 2024-08-15T18:02:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30586.log 2024-08-15T18:02:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34511.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29695.log.gz 2024-08-15T18:02:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34036.log 2024-08-15T18:02:24.239 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34511.log.gz 2024-08-15T18:02:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30235.log 2024-08-15T18:02:24.240 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33550.log.gz 2024-08-15T18:02:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30586.log: /var/log/ceph/ceph-client.admin.34036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30264.log 2024-08-15T18:02:24.240 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30586.log.gz 2024-08-15T18:02:24.240 INFO:teuthology.orchestra.run.smithi038.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34036.log.gz 2024-08-15T18:02:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31760.log 2024-08-15T18:02:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34117.log 2024-08-15T18:02:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30264.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30235.log.gz 2024-08-15T18:02:24.241 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30040.log 2024-08-15T18:02:24.241 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31760.log: /var/log/ceph/ceph-client.admin.34117.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30264.log.gz 2024-08-15T18:02:24.241 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31515.log 2024-08-15T18:02:24.241 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34117.log.gz 2024-08-15T18:02:24.241 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-08-15T18:02:24.241 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30040.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31760.log.gz 2024-08-15T18:02:24.241 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30775.log 2024-08-15T18:02:24.241 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31515.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30040.log.gz 2024-08-15T18:02:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33496.log 2024-08-15T18:02:24.242 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31515.log.gz 2024-08-15T18:02:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34592.log 2024-08-15T18:02:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33442.log 2024-08-15T18:02:24.242 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2%/var/log/ceph/ceph-client.admin.33496.log: -- replaced with /var/log/ceph/ceph-client.admin.30775.log.gz 2024-08-15T18:02:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33147.log 2024-08-15T18:02:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34592.log: 92.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-08-15T18:02:24.242 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29827.log 2024-08-15T18:02:24.243 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33496.log.gz 2024-08-15T18:02:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33442.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34592.log.gz 2024-08-15T18:02:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-08-15T18:02:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33147.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33442.log.gz 2024-08-15T18:02:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29827.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29827.log.gz 2024-08-15T18:02:24.243 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33147.log.gz 2024-08-15T18:02:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32235.log 2024-08-15T18:02:24.243 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-mds.a.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-08-15T18:02:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34484.log 2024-08-15T18:02:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33577.log 2024-08-15T18:02:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32235.log: /var/log/ceph/ceph-client.admin.34484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32416.log 2024-08-15T18:02:24.244 INFO:teuthology.orchestra.run.smithi038.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32235.log.gz 2024-08-15T18:02:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33577.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34484.log.gz 2024-08-15T18:02:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32524.log 2024-08-15T18:02:24.244 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33577.log.gz 2024-08-15T18:02:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30250.log 2024-08-15T18:02:24.244 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34376.log 2024-08-15T18:02:24.245 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32416.log.gz 2024-08-15T18:02:24.245 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32524.log.gz 2024-08-15T18:02:24.245 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30250.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30250.log.gz 2024-08-15T18:02:24.245 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30045.log 2024-08-15T18:02:24.245 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30263.log 2024-08-15T18:02:24.245 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29813.log 2024-08-15T18:02:24.245 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34295.log 2024-08-15T18:02:24.246 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30263.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34376.log.gz 2024-08-15T18:02:24.246 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33685.log 2024-08-15T18:02:24.246 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30045.log.gz 2024-08-15T18:02:24.246 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30263.log.gz 2024-08-15T18:02:24.246 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29813.log.gz 2024-08-15T18:02:24.246 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31789.log 2024-08-15T18:02:24.246 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34295.log: /var/log/ceph/ceph-client.admin.33685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29156.log 2024-08-15T18:02:24.246 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34295.log.gz 2024-08-15T18:02:24.247 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33685.log.gz 2024-08-15T18:02:24.247 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33847.log 2024-08-15T18:02:24.247 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31789.log: /var/log/ceph/ceph-client.admin.29156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32663.log 2024-08-15T18:02:24.247 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31789.log.gz 2024-08-15T18:02:24.247 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33847.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29156.log.gz 2024-08-15T18:02:24.247 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33793.log 2024-08-15T18:02:24.247 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32663.log: gzip -5 --verbose -- 87.6%/var/log/ceph/ceph-client.admin.29821.log 2024-08-15T18:02:24.247 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33847.log.gz 2024-08-15T18:02:24.248 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32663.log.gz 2024-08-15T18:02:24.248 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30721.log 2024-08-15T18:02:24.248 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33793.log: /var/log/ceph/ceph-client.admin.29821.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-08-15T18:02:24.248 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33793.log.gz 2024-08-15T18:02:24.248 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29826.log 2024-08-15T18:02:24.248 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30721.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29821.log.gz 2024-08-15T18:02:24.248 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-mgr.y.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30721.log.gz-5 2024-08-15T18:02:24.248 INFO:teuthology.orchestra.run.smithi038.stderr:--verbose -- /var/log/ceph/ceph-client.admin.34009.log 2024-08-15T18:02:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23995.log 2024-08-15T18:02:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29830.log 2024-08-15T18:02:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34009.log: /var/log/ceph/ceph-client.admin.23995.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29826.log.gz 2024-08-15T18:02:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30829.log 2024-08-15T18:02:24.249 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34009.log.gz 2024-08-15T18:02:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29830.log: gzip 89.1%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.23995.log.gz-- 2024-08-15T18:02:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34837.log 2024-08-15T18:02:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32879.log 2024-08-15T18:02:24.250 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30829.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29830.log.gz 2024-08-15T18:02:24.250 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose 88.6%-- /var/log/ceph/ceph-client.admin.32154.log -- replaced with /var/log/ceph/ceph-client.admin.30829.log.gz 2024-08-15T18:02:24.250 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-08-15T18:02:24.250 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34266.log 2024-08-15T18:02:24.250 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32879.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34837.log.gz 2024-08-15T18:02:24.250 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32360.log 2024-08-15T18:02:24.250 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32879.log.gz 2024-08-15T18:02:24.251 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2%/var/log/ceph/ceph-client.admin.34266.log: -- replaced with /var/log/ceph/ceph-client.admin.32154.log.gzgzip 2024-08-15T18:02:24.251 INFO:teuthology.orchestra.run.smithi038.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.32960.log 2024-08-15T18:02:24.251 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29338.log 2024-08-15T18:02:24.251 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32360.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34266.log.gz 2024-08-15T18:02:24.251 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30258.log 2024-08-15T18:02:24.251 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32360.log.gz 2024-08-15T18:02:24.251 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32605.log 2024-08-15T18:02:24.251 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29338.log: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-08-15T18:02:24.251 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33523.log 2024-08-15T18:02:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30258.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32605.log: /var/log/ceph/ceph-client.admin.30244.log 2024-08-15T18:02:24.252 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29338.log.gz 91.1% -- replaced with /var/log/ceph/ceph-client.admin.32960.log.gz 2024-08-15T18:02:24.252 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-08-15T18:02:24.252 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30258.log.gz 2024-08-15T18:02:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29533.log 2024-08-15T18:02:24.252 INFO:teuthology.orchestra.run.smithi038.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32605.log.gz 2024-08-15T18:02:24.253 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33523.log.gz 2024-08-15T18:02:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31677.log 2024-08-15T18:02:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29824.log 2024-08-15T18:02:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29533.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30244.log.gz 2024-08-15T18:02:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34063.log 2024-08-15T18:02:24.253 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29533.log.gz 2024-08-15T18:02:24.253 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.31677.log.gz 2024-08-15T18:02:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30559.log 2024-08-15T18:02:24.254 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29479.log 2024-08-15T18:02:24.254 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34063.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29824.log.gz 2024-08-15T18:02:24.254 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30559.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34063.log.gz 2024-08-15T18:02:24.254 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30559.log.gz 2024-08-15T18:02:24.254 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34239.log 2024-08-15T18:02:24.254 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-15T18:02:24.254 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29479.log: /var/log/ceph/ceph-client.admin.34239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32852.log 2024-08-15T18:02:24.254 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29479.log.gz 2024-08-15T18:02:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33982.log 2024-08-15T18:02:24.255 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34239.log.gz 2024-08-15T18:02:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.32852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32100.log 2024-08-15T18:02:24.255 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32852.log.gz 2024-08-15T18:02:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29722.log 2024-08-15T18:02:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33982.log: /var/log/ceph/ceph-client.admin.32100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33469.log 2024-08-15T18:02:24.255 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.33982.log.gz-5 2024-08-15T18:02:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:--verbose -- /var/log/ceph/ceph-client.admin.31071.log 2024-08-15T18:02:24.256 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8%/var/log/ceph/ceph-client.admin.29722.log: -- replaced with /var/log/ceph/ceph-client.admin.32100.log.gz 2024-08-15T18:02:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30802.log 2024-08-15T18:02:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33469.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29722.log.gz 2024-08-15T18:02:24.256 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-15T18:02:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30451.log 2024-08-15T18:02:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31071.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33469.log.gz 2024-08-15T18:02:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30245.log 2024-08-15T18:02:24.256 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31071.log.gz 2024-08-15T18:02:24.257 INFO:teuthology.orchestra.run.smithi038.stderr:gzip /var/log/ceph/ceph-client.admin.30451.log: 88.2%-5 -- replaced with /var/log/ceph/ceph-client.admin.30802.log.gz--verbose 2024-08-15T18:02:24.257 INFO:teuthology.orchestra.run.smithi038.stderr:-- /var/log/ceph/ceph-client.admin.31965.log 2024-08-15T18:02:24.257 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31380.log 2024-08-15T18:02:24.257 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30245.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30451.log.gz 2024-08-15T18:02:24.257 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32073.log 2024-08-15T18:02:24.257 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30245.log.gz 2024-08-15T18:02:24.257 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31965.log.gz 2024-08-15T18:02:24.257 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29587.log 2024-08-15T18:02:24.258 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31024.log 2024-08-15T18:02:24.258 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32073.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31380.log.gz 2024-08-15T18:02:24.258 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30042.log 2024-08-15T18:02:24.258 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32073.log.gz 2024-08-15T18:02:24.258 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.31024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30047.log 2024-08-15T18:02:24.258 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29587.log.gz 2024-08-15T18:02:24.258 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31623.log 2024-08-15T18:02:24.259 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30042.log.gz 2024-08-15T18:02:24.259 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32208.log 2024-08-15T18:02:24.259 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.31024.log.gz 2024-08-15T18:02:24.259 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31299.log 2024-08-15T18:02:24.259 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31623.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30047.log.gz 2024-08-15T18:02:24.259 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32127.log 2024-08-15T18:02:24.259 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31623.log.gz 2024-08-15T18:02:24.259 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30046.log 2024-08-15T18:02:24.259 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31299.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32208.log.gz 2024-08-15T18:02:24.260 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31212.log 2024-08-15T18:02:24.260 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32127.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31299.log.gz 2024-08-15T18:02:24.260 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33712.log 2024-08-15T18:02:24.260 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32127.log.gz 2024-08-15T18:02:24.260 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31212.log: gzip -5 89.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.30046.log.gz/var/log/ceph/ceph-client.admin.34889.log 2024-08-15T18:02:24.260 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-08-15T18:02:24.260 INFO:teuthology.orchestra.run.smithi038.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31212.log.gz 2024-08-15T18:02:24.260 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30912.log 2024-08-15T18:02:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31596.log 2024-08-15T18:02:24.261 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33712.log.gz 2024-08-15T18:02:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34457.log 2024-08-15T18:02:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30912.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34889.log.gz 2024-08-15T18:02:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30667.log 2024-08-15T18:02:24.261 INFO:teuthology.orchestra.run.smithi038.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30912.log.gz 2024-08-15T18:02:24.261 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31596.log.gz 2024-08-15T18:02:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31569.log 2024-08-15T18:02:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34457.log: /var/log/ceph/ceph-client.admin.30667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31461.log 2024-08-15T18:02:24.262 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34457.log.gz 2024-08-15T18:02:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31569.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30667.log.gz-5 2024-08-15T18:02:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:--verbose -- /var/log/ceph/ceph-client.admin.32578.log 2024-08-15T18:02:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31461.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31569.log.gz 2024-08-15T18:02:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32690.log 2024-08-15T18:02:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34144.log 2024-08-15T18:02:24.262 INFO:teuthology.orchestra.run.smithi038.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31461.log.gz 2024-08-15T18:02:24.263 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32578.log.gz 2024-08-15T18:02:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24119.log /var/log/ceph/ceph-client.admin.32690.log: 2024-08-15T18:02:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31239.log 2024-08-15T18:02:24.263 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32690.log.gz 2024-08-15T18:02:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31650.log 2024-08-15T18:02:24.263 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34144.log.gz 2024-08-15T18:02:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.24119.log: /var/log/ceph/ceph-client.admin.31239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33955.log 2024-08-15T18:02:24.263 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.24119.log.gz 2024-08-15T18:02:24.264 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31239.log.gz 2024-08-15T18:02:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31434.log 2024-08-15T18:02:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31650.log: gzip /var/log/ceph/ceph-client.admin.33955.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34198.log 2024-08-15T18:02:24.264 INFO:teuthology.orchestra.run.smithi038.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31650.log.gz 2024-08-15T18:02:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34862.log 2024-08-15T18:02:24.264 INFO:teuthology.orchestra.run.smithi038.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33955.log.gz 2024-08-15T18:02:24.264 INFO:teuthology.orchestra.run.smithi038.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31434.log.gz 2024-08-15T18:02:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29256.log 2024-08-15T18:02:24.265 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34198.log.gz 2024-08-15T18:02:24.265 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33604.log 2024-08-15T18:02:24.265 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34862.log: /var/log/ceph/ceph-client.admin.29256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29311.log 2024-08-15T18:02:24.265 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34862.log.gz 2024-08-15T18:02:24.265 INFO:teuthology.orchestra.run.smithi038.stderr: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.29256.log.gz 2024-08-15T18:02:24.265 INFO:teuthology.orchestra.run.smithi038.stderr:-5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-08-15T18:02:24.265 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33604.log: /var/log/ceph/ceph-client.admin.29311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33261.log 2024-08-15T18:02:24.265 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33604.log.gz 2024-08-15T18:02:24.266 INFO:teuthology.orchestra.run.smithi038.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29311.log.gz 2024-08-15T18:02:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:gzip /var/log/ceph/ceph-osd.1.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29419.log 2024-08-15T18:02:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29365.log 2024-08-15T18:02:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33174.log 2024-08-15T18:02:24.266 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33261.log.gz 2024-08-15T18:02:24.266 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29419.log.gz 2024-08-15T18:02:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31272.log 2024-08-15T18:02:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29365.log: /var/log/ceph/ceph-client.admin.33174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32181.log 2024-08-15T18:02:24.267 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29365.log.gz 2024-08-15T18:02:24.267 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33315.log 2024-08-15T18:02:24.267 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33174.log.gz 2024-08-15T18:02:24.267 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31272.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31272.log.gz 2024-08-15T18:02:24.267 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33631.log 2024-08-15T18:02:24.267 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33011.log 2024-08-15T18:02:24.267 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33315.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32181.log.gz 2024-08-15T18:02:24.268 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32046.log 2024-08-15T18:02:24.268 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33631.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33315.log.gz 2024-08-15T18:02:24.268 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32387.log 2024-08-15T18:02:24.268 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33011.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33631.log.gz 2024-08-15T18:02:24.268 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33069.log 2024-08-15T18:02:24.268 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32046.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33011.log.gz 2024-08-15T18:02:24.268 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32470.log 2024-08-15T18:02:24.268 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32387.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32046.log.gz 2024-08-15T18:02:24.269 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.33069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29641.log 2024-08-15T18:02:24.269 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32387.log.gz 2024-08-15T18:02:24.269 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31706.log 2024-08-15T18:02:24.269 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.31162.log 2024-08-15T18:02:24.269 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32551.log 2024-08-15T18:02:24.269 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32470.log.gz 2024-08-15T18:02:24.269 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31706.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.29641.log.gz 2024-08-15T18:02:24.269 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.2.31162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32497.log 2024-08-15T18:02:24.269 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31706.log.gz 2024-08-15T18:02:24.270 INFO:teuthology.orchestra.run.smithi038.stderr:gzip /var/log/ceph/ceph-client.admin.32551.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34727.log 2024-08-15T18:02:24.270 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29560.log 2024-08-15T18:02:24.270 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32497.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32551.log.gz 2024-08-15T18:02:24.270 INFO:teuthology.orchestra.run.smithi038.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.2.31162.log.gz 2024-08-15T18:02:24.270 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34673.log 2024-08-15T18:02:24.270 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34727.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32497.log.gz 2024-08-15T18:02:24.270 INFO:teuthology.orchestra.run.smithi038.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.33069.log.gz 2024-08-15T18:02:24.270 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32717.log 2024-08-15T18:02:24.271 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29560.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34727.log.gz 2024-08-15T18:02:24.271 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32906.log 2024-08-15T18:02:24.271 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34673.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29560.log.gz 2024-08-15T18:02:24.271 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32443.log 2024-08-15T18:02:24.271 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32717.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34673.log.gz 2024-08-15T18:02:24.271 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34619.log 2024-08-15T18:02:24.271 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32717.log.gz 2024-08-15T18:02:24.272 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32906.log.gz 2024-08-15T18:02:24.272 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31488.log 2024-08-15T18:02:24.272 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31407.log 2024-08-15T18:02:24.272 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34619.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32443.log.gz 2024-08-15T18:02:24.272 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32933.log 2024-08-15T18:02:24.272 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31488.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34619.log.gz 2024-08-15T18:02:24.272 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31911.log 2024-08-15T18:02:24.273 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31407.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31488.log.gz 2024-08-15T18:02:24.273 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33288.log 2024-08-15T18:02:24.273 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31407.log.gz 2024-08-15T18:02:24.273 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29614.log 2024-08-15T18:02:24.273 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32933.log.gz 2024-08-15T18:02:24.273 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33766.log 2024-08-15T18:02:24.273 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33288.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31911.log.gz 2024-08-15T18:02:24.273 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34781.log 2024-08-15T18:02:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29614.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33288.log.gz 2024-08-15T18:02:24.274 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.29614.log.gz-5 2024-08-15T18:02:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:--verbose -- /var/log/ceph/ceph-client.admin.32634.log 2024-08-15T18:02:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30041.log 2024-08-15T18:02:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34781.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33766.log.gz 2024-08-15T18:02:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32825.log 2024-08-15T18:02:24.274 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34781.log.gz 2024-08-15T18:02:24.275 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32634.log.gz/var/log/ceph/ceph-client.admin.30041.log: 2024-08-15T18:02:24.275 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34430.log 2024-08-15T18:02:24.275 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32771.log 2024-08-15T18:02:24.275 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32825.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30041.log.gz 2024-08-15T18:02:24.275 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33415.log 2024-08-15T18:02:24.275 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34430.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32825.log.gz 2024-08-15T18:02:24.275 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31992.log 2024-08-15T18:02:24.275 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32771.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34430.log.gz 2024-08-15T18:02:24.276 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31326.log 2024-08-15T18:02:24.276 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33415.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32771.log.gz 2024-08-15T18:02:24.276 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29749.log 2024-08-15T18:02:24.276 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33415.log.gz 2024-08-15T18:02:24.276 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30505.log 2024-08-15T18:02:24.276 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31992.log.gz 2024-08-15T18:02:24.276 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33739.log 2024-08-15T18:02:24.276 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29749.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31326.log.gz 2024-08-15T18:02:24.276 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31938.log 2024-08-15T18:02:24.277 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29749.log.gz 2024-08-15T18:02:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34090.log 2024-08-15T18:02:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33739.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30505.log.gz 2024-08-15T18:02:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-15T18:02:24.277 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33739.log.gz 2024-08-15T18:02:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31938.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31938.log.gz 2024-08-15T18:02:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31353.log 2024-08-15T18:02:24.277 INFO:teuthology.orchestra.run.smithi038.stderr:gzip /var/log/ceph/ceph.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29190.log 2024-08-15T18:02:24.278 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34090.log.gz 2024-08-15T18:02:24.278 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31353.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.19137.log 2024-08-15T18:02:24.278 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31353.log.gz 2024-08-15T18:02:24.278 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-08-15T18:02:24.278 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29190.log: 84.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-15T18:02:24.278 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32744.log 2024-08-15T18:02:24.278 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph.tmp-client.admin.19137.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.19137.log.gz 2024-08-15T18:02:24.278 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29190.log.gz 2024-08-15T18:02:24.279 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31884.log 2024-08-15T18:02:24.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33820.log 2024-08-15T18:02:24.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32744.log: gzip /var/log/ceph/ceph-client.admin.31884.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29283.log 2024-08-15T18:02:24.279 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32744.log.gz 2024-08-15T18:02:24.279 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31884.log.gz 2024-08-15T18:02:24.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33820.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33820.log.gz 2024-08-15T18:02:24.279 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33201.log 2024-08-15T18:02:24.279 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.29283.log: -- /var/log/ceph/ceph-client.admin.31816.log 2024-08-15T18:02:24.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32268.log 2024-08-15T18:02:24.280 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29283.log.gz 2024-08-15T18:02:24.280 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8%/var/log/ceph/ceph-client.admin.31816.log: -- replaced with /var/log/ceph/ceph-client.admin.33201.log.gz 2024-08-15T18:02:24.280 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31733.log 2024-08-15T18:02:24.280 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 87.9%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.31816.log.gz-- 2024-08-15T18:02:24.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33901.log 2024-08-15T18:02:24.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30856.log 2024-08-15T18:02:24.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.31733.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32268.log.gz 2024-08-15T18:02:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30748.log 2024-08-15T18:02:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33901.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31733.log.gz 2024-08-15T18:02:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30856.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33901.log.gz 2024-08-15T18:02:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32798.log 2024-08-15T18:02:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34565.log 2024-08-15T18:02:24.281 INFO:teuthology.orchestra.run.smithi038.stderr: 91.1%/var/log/ceph/ceph-client.admin.32798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29668.log 2024-08-15T18:02:24.281 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30856.log.gz 2024-08-15T18:02:24.281 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30748.log.gz 2024-08-15T18:02:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34565.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32798.log.gz 2024-08-15T18:02:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29776.log 2024-08-15T18:02:24.282 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34565.log.gz 2024-08-15T18:02:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30694.log 2024-08-15T18:02:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29668.log: /var/log/ceph/ceph-client.admin.29776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34808.log 2024-08-15T18:02:24.282 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29668.log.gz 2024-08-15T18:02:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30694.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-08-15T18:02:24.282 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29776.log.gz 2024-08-15T18:02:24.283 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30694.log.gz 2024-08-15T18:02:24.283 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34349.log 2024-08-15T18:02:24.283 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34808.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34754.log 2024-08-15T18:02:24.283 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34808.log.gz 2024-08-15T18:02:24.283 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33928.log 2024-08-15T18:02:24.283 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34349.log: /var/log/ceph/ceph-client.admin.34754.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34349.log.gz 2024-08-15T18:02:24.283 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34754.log.gz 2024-08-15T18:02:24.283 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33874.log 2024-08-15T18:02:24.284 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30251.log 2024-08-15T18:02:24.284 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33874.log: gzip 87.6%-5 -- replaced with /var/log/ceph/ceph-client.admin.33928.log.gz 2024-08-15T18:02:24.284 INFO:teuthology.orchestra.run.smithi038.stderr:--verbose -- /var/log/ceph/ceph-client.admin.30478.log 2024-08-15T18:02:24.284 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33342.log 2024-08-15T18:02:24.284 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30251.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33874.log.gz 2024-08-15T18:02:24.284 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.30478.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.30251.log.gz 2024-08-15T18:02:24.285 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30478.log.gz 2024-08-15T18:02:24.285 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29831.log 2024-08-15T18:02:24.285 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34171.log 2024-08-15T18:02:24.285 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29099.log 2024-08-15T18:02:24.285 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33342.log.gz 2024-08-15T18:02:24.285 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34646.log 2024-08-15T18:02:24.285 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34171.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29831.log.gz 2024-08-15T18:02:24.285 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.29099.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34171.log.gz 2024-08-15T18:02:24.285 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29099.log.gz 2024-08-15T18:02:24.286 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29829.log 2024-08-15T18:02:24.286 INFO:teuthology.orchestra.run.smithi038.stderr:gzip /var/log/ceph/ceph-client.admin.34646.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33120.log 2024-08-15T18:02:24.286 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29829.log: /var/log/ceph/ceph-client.admin.29506.log 2024-08-15T18:02:24.286 INFO:teuthology.orchestra.run.smithi038.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34646.log.gz 2024-08-15T18:02:24.286 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32333.log 2024-08-15T18:02:24.286 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33120.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29829.log.gz 2024-08-15T18:02:24.286 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34403.log 2024-08-15T18:02:24.286 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4%/var/log/ceph/ceph-client.admin.29506.log: -- replaced with /var/log/ceph/ceph-client.admin.33120.log.gz 2024-08-15T18:02:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34700.log 2024-08-15T18:02:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.32333.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29506.log.gz 2024-08-15T18:02:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33228.log 2024-08-15T18:02:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34403.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32333.log.gz 2024-08-15T18:02:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34700.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34403.log.gz 2024-08-15T18:02:24.287 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34700.log.gz 2024-08-15T18:02:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-08-15T18:02:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.33228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34322.log 2024-08-15T18:02:24.288 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1%/var/log/ceph/ceph-osd.2.log: -- replaced with /var/log/ceph/ceph-client.admin.33228.log.gz 2024-08-15T18:02:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34322.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34322.log.gz 2024-08-15T18:02:24.288 INFO:teuthology.orchestra.run.smithi104.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-08-15T18:02:24.292 INFO:teuthology.orchestra.run.smithi104.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-08-15T18:02:24.320 INFO:teuthology.orchestra.run.smithi104.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-08-15T18:02:24.326 INFO:teuthology.orchestra.run.smithi104.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-08-15T18:02:24.360 INFO:teuthology.orchestra.run.smithi104.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-08-15T18:02:24.386 INFO:teuthology.orchestra.run.smithi038.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-08-15T18:02:24.393 INFO:teuthology.orchestra.run.smithi038.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-08-15T18:02:24.396 INFO:teuthology.orchestra.run.smithi038.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-08-15T18:02:24.409 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-08-15T18:02:24.411 INFO:teuthology.orchestra.run.smithi038.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-08-15T18:02:24.413 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-08-15T18:02:24.413 INFO:teuthology.orchestra.run.smithi038.stderr:real 0m0.300s 2024-08-15T18:02:24.413 INFO:teuthology.orchestra.run.smithi038.stderr:user 0m1.200s 2024-08-15T18:02:24.413 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m0.126s 2024-08-15T18:02:24.415 INFO:teuthology.orchestra.run.smithi104.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-08-15T18:02:24.418 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-08-15T18:02:24.418 INFO:teuthology.orchestra.run.smithi104.stderr:real 0m0.303s 2024-08-15T18:02:24.418 INFO:teuthology.orchestra.run.smithi104.stderr:user 0m1.234s 2024-08-15T18:02:24.418 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m0.059s 2024-08-15T18:02:24.419 INFO:tasks.ceph:Archiving logs... 2024-08-15T18:02:24.419 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038/remote/smithi038/log 2024-08-15T18:02:24.420 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-15T18:02:24.767 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038/remote/smithi104/log 2024-08-15T18:02:24.768 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-15T18:02:24.953 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-15T18:02:24.977 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-15T18:02:24.977 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-15T18:02:24.980 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-15T18:02:25.054 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-15T18:02:25.063 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-15T18:02:25.063 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T18:02:25.065 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T18:02:25.089 INFO:teuthology.orchestra.run.smithi038.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T18:02:25.089 INFO:teuthology.orchestra.run.smithi038.stdout:============================================================================== 2024-08-15T18:02:25.090 INFO:teuthology.orchestra.run.smithi038.stdout:+hv01.front.sepi 67.205.162.81 3 u 57 64 177 0.091 -1.477 0.654 2024-08-15T18:02:25.090 INFO:teuthology.orchestra.run.smithi038.stdout:*hv02.front.sepi 63.231.80.2 3 u 54 64 177 0.081 0.504 0.780 2024-08-15T18:02:25.090 INFO:teuthology.orchestra.run.smithi038.stdout:+hv03.front.sepi 74.6.168.72 3 u 53 64 177 0.077 1.385 0.818 2024-08-15T18:02:25.090 INFO:teuthology.orchestra.run.smithi038.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T18:02:25.092 INFO:teuthology.orchestra.run.smithi104.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T18:02:25.092 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================== 2024-08-15T18:02:25.092 INFO:teuthology.orchestra.run.smithi104.stdout:+hv01.front.sepi 67.205.162.81 3 u 54 64 177 0.113 0.582 1.489 2024-08-15T18:02:25.092 INFO:teuthology.orchestra.run.smithi104.stdout:*hv02.front.sepi 63.231.80.2 3 u 51 64 177 0.104 2.696 1.257 2024-08-15T18:02:25.092 INFO:teuthology.orchestra.run.smithi104.stdout:+hv03.front.sepi 74.6.168.72 3 u 49 64 177 0.121 2.775 1.457 2024-08-15T18:02:25.092 INFO:teuthology.orchestra.run.smithi104.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T18:02:25.093 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-15T18:02:25.101 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-15T18:02:25.102 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-15T18:02:25.110 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-15T18:02:25.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-15T18:02:25.127 INFO:teuthology.task.internal:Duration was 795.389151 seconds 2024-08-15T18:02:25.127 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-15T18:02:25.136 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-15T18:02:25.136 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-15T18:02:25.139 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-15T18:02:25.190 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-15T18:02:25.191 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-08-15T18:02:25.191 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-15T18:02:25.203 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-08-15T18:02:25.204 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-15T18:02:25.217 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-15T18:02:25.217 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-15T18:02:25.249 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-15T18:02:25.289 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-15T18:02:25.290 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-15T18:02:25.293 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-15T18:02:25.408 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-15T18:02:25.418 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-15T18:02:25.419 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-15T18:02:25.434 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-15T18:02:25.452 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-15T18:02:25.463 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-15T18:02:25.465 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-15T18:02:25.478 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-08-15T18:02:25.508 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-08-15T18:02:25.520 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-15T18:02:25.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T18:02:25.526 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-15T18:02:25.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T18:02:25.532 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-15T18:02:25.542 INFO:teuthology.task.internal:Transferring archived files... 2024-08-15T18:02:25.542 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038/remote/smithi038 2024-08-15T18:02:25.543 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-15T18:02:25.593 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-15_16:39:12-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7857038/remote/smithi104 2024-08-15T18:02:25.593 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-15T18:02:25.619 INFO:teuthology.task.internal:Removing archive directory... 2024-08-15T18:02:25.619 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-15T18:02:25.622 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-15T18:02:25.627 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-15T18:02:25.637 INFO:teuthology.task.internal:Not uploading archives. 2024-08-15T18:02:25.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-15T18:02:25.647 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-15T18:02:25.647 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-15T18:02:25.650 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-15T18:02:25.655 INFO:teuthology.orchestra.run.smithi038.stdout: 924110 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 15 18:02 /home/ubuntu/cephtest 2024-08-15T18:02:25.657 INFO:teuthology.orchestra.run.smithi104.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 15 18:02 /home/ubuntu/cephtest 2024-08-15T18:02:25.659 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-15T18:02:25.669 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-15T18:02:25.691 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} duration: 795.3891513347626 failure_reason: 'Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=434350764f0e471da729700c3aa8f152 status: fail success: false 2024-08-15T18:02:25.692 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-15T18:02:25.767 INFO:teuthology.run:FAIL